ztp/docs/05-connecting-and-saving.md

+++
title = "Databases: Connecting and Using"
date = 2023-03-20T17:38:12Z
weight = 4
+++

## Chapter 3.8: Databases

First, we're gonna expand the configuration we defined previously.  The one
thing we will require is the password, although eventually I'm going to make
that a command-line option using Clap.

``` rust
#[derive(serde::Deserialize)]
#[serde(default)]
pub struct DatabaseSettings {
    pub username: String,
    pub password: String,
    pub host: String,
    pub port: u16,
    pub database: String,
}

#[derive(serde::Deserialize)]
#[serde(default)]
pub struct Settings {
    pub database: DatabaseSettings,
    pub port: u16,
}

impl Default for DatabaseSettings {
    fn default() -> Self {
        DatabaseSettings {
            username: "newsletter".to_string(),
            password: "".to_string(),
            host: "localhost".to_string(),
            port: 5432,
            database: "newsletter".to_string(),
        }
    }
}

impl Default for Settings {
    fn default() -> Self {
        Settings {
            port: 3001,
            database: DatabaseSettings::default(),
        }
    }
}
```

## Cleanup on Aisle Three

The code's beginning to get a bit messsy. So let's re-arrange. I already have
the configuration handler in its own file, but let's clean up the routes. I'm
going to take the functions I've already defined and put them in a subfolder,
`src/routes`, and then for each one I'll make a new file.  For example:

``` rust
use axum::{http::StatusCode, response::IntoResponse};

pub(crate) async fn health_check() -> impl IntoResponse {
    (StatusCode::OK, ())
}
```

Just a note: *ignore* your IDE's "advice" to remove the unneeded `async`; Axum
will not compile this module correctly, `IntoResponse` requires that it be an
async function. I don't know why rust-analyzer failed to pick that up.

Now make a `src/routes.rs` (yes, the same name as the folder).  You can activate
and de-activate routes at will:

``` rust
use axum::{
    routing::{get, post},
    Router,
};

mod greet;
use greet::greet;

mod health_check;
use health_check::health_check;

mod index;
use index::index;

mod subscribe;
use subscribe::subscribe;

pub(crate) fn app() -> Router {
    Router::new()
        .route("/", get(index))
        .route("/subscriptions", post(subscribe))
        .route("/:name", get(greet))
        .route("/health_check", get(health_check))
}
```

You don't need the `use` clauses, you could just say `index::index`, but I kinda
like the way this looks, even with the repetition.  It makes it clear what I'm
doing and what I'm looking at.  This is Rust 2021, where the presence of a file
and a folder with the same name indicate a module with submodules.  In prior
editions of rust, instead of `routes.rs`, we would have used `routes/mod.rs`.
I'm not entirely sure which I like better.

Now that that's done, in our `lib.rs`, we use the `routes`:

``` rust
use std::net::SocketAddr;
mod configuration;
use configuration::get_configuration;
mod routes;
use routes::app;

pub async fn run() {
    let configuration = get_configuration().unwrap();
    let addr = SocketAddr::from(([127, 0, 0, 1], configuration.port));
    tracing::info!("listening on {}", addr);
    axum::Server::bind(&addr)
        .serve(app().into_make_service())
        .await
        .unwrap()
}

```

All of the Axum imports have been removed, since they're part of the routes.
Nice, clean and clear code.  At this point, even `cargo clippy` is happy.

### The Database String

Connecting to a database these days involves a URL.  Let's make it easy to
generate one.  Now, we could implement the trait `std::string::ToString`, or
`Display`, but that would conflict with any debugging information.  Instead,
let's define a new method in `configuration.rs`:

``` rust
impl DatabaseSettings {
    pub fn url(&self) -> String {
        if self.password.len() == 0 {
            format!(
                "postgres://{}@{}:{}/{}",
                self.username, self.host, self.port, self.database
            )
        } else {
            format!(
                "postgres://{}:{}@{}:{}/{}",
                self.username, self.password, self.host, self.port, self.database
            )
        }
    }
}
```

It is possible (unwise, but possible) to connect to a Postgres database with no
password.  In a dynamic language this would have been less wordy, but I can't
complain about Rust being a little extra verbose in exchange for being a lot
more precise.

### Providing the database to the application

Since we're using Postgres, in `lib.rs` we'll add `use
sqlx::postgres::PgPoolOptions;`, which will allow us to create a pool of workers
and limit how many there are.  In the `run()` function, we're going to extract
the `app()` definition and use it:

``` rust
use axum::Extension;
use sqlx::postgres::PgPoolOptions;

// in: fn run()

    let pool = PgPoolOptions::new()
        .max_connections(50)
        .connect(&configuration.database.url())
        .await
        .expect("could not connect to database_url");

    let routes = app().layer(Extension(pool));

    tracing::info!("listening on {}", addr);
    axum::Server::bind(&addr)
        .serve(routes.into_make_service())
        .await
        .unwrap()
```

Now we have a configuration that is runnable.  If you run it as-is, it will fail
because, if you set up your Postgres database correctly, there is no password.
Create a `ztp.config.yaml` file (or JSON, or TOML, or whatever floats your boat)
and establish the password:

``` yaml
database:
    password: redacted
```

At one point, I had this file named only `ztp.config`, and unfortunately it
crashed.  Adding `#[derive(Debug)]` to my configuration structs helped show that
the password wasn't getting set, but it took me a few minutes to realize that
I'd forgotten the extension so `Config` didn't know how to read the file. Naming
it `ztp.config.yaml` fixed it, and this illustrates just how useful TOML can be
in circumstances like this.

### Pulling in a subscription

This turned out to be a bit of a beast, and I'm incredibly indebted to Carlos
Armando Marcano Vargas and [his blog](https://github.com/carlosm27/blog) where
he documented much of what I needed to make this work with Axum.

In the migrations file, we specified that the primary key of our subscriptions
table would be a UUID, and that we were going to record when the subscription
happened.  To do this, first we have to make sure that we have the `Uuid` and
`Chrono` crates, and that they're all hooked up for proper serialization:

``` toml
chrono = { version = "0.4.24", features = ["serde"] }
uuid = { version = "1.3.0", features = ["v4", "serde"] }
sqlx = { version = "0.6.3", features = ["runtime-tokio-native-tls", "macros", 
         "postgres", "uuid", "chrono"] }
```

Both of the new creates need `serde` as a feature and we'll be using `UUID4` for
our primary key.

We'll also need a full subscription object to store, and then a
way to generate it from a new subscription:

```
#[derive(serde::Deserialize)]
pub(crate) struct NewSubscription {
    pub email: String,
    pub name: String,
}

#[derive(serde::Deserialize, serde::Serialize, sqlx::FromRow)]
struct Subscription {
    pub id: Uuid,
    pub email: String,
    pub name: String,
    pub subscribed_at: DateTime<Utc>,
}

impl From<&NewSubscription> for Subscription {
    fn from(s: &NewSubscription) -> Self {
        Subscription {
            id: Uuid::new_v4(),
            email: s.email.clone(),
            name: s.name.clone(),
            subscribed_at: Utc::now(),
        }
    }
}
```

I've renamed "Subscription" to "NewSubscription" to distinguish it from the
thing we're going to reliably keep in our database, which has all the fields.

And now we're going to re-write the `subscribe()` function to actually talk to
the database.  All of those parens around the `payload` are necessary to help
Rust understand what we want to borrow.  You'll note that it's `payload.0`, not
just `payload`; Form objects can have multiple instances of their content.


```
pub(crate) async fn subscribe(Extension(pool): Extension<PgPool>, payload: Form<NewSubscription>) -> impl IntoResponse {
    let sql = "INSERT INTO subscriptions (id, email, name, subscribed_at) VALUES ($1, $2, $3, $4);".to_string();
    let subscription: Subscription = (&(payload.0)).into();

    let _ = sqlx::query(&sql)
        .bind(subscription.id)
        .bind(subscription.email)
        .bind(subscription.name)
        .bind(subscription.subscribed_at)
        .execute(&pool)
        .await
        .expect("Failed for some reason?");

    (StatusCode::OK, ())
}
```

Alternatively, I could have written:

``` rust
    let subscription = Subscription::from(&(payload.0));
```

Either way works.

### Better error handling.

That unused `.expect()` has Tokio panicking if I give it a duplicate email
address, as one of the constraints in our original migration reads:

``` sql
   email text not null unique,
```

We want to catch and handle that error.  We also want to fix something that's
been sitting in the "TODO" list for awhile: make sure that when the subscription
form isn't fully filled out, we return a straight 400.  That's actually the
easier part: we make the payload optional and return our default `BAD_REQUEST`
if it's not there:

```
pub(crate) async fn subscribe(
    Extension(pool): Extension<PgPool>,
    payload: Option<Form<NewSubscription>>,
) -> impl IntoResponse {
    if let Some(payload) = payload {
    ...
        (StatusCode::OK, ())
    } else {
        (StatusCode::BAD_REQUEST, ())
    }
```    

So, handling the errors is going to be interesting.  Going above and beyond,
it's time to use `thiserror`.