+++ title = "Databases: Connecting and Using" date = 2023-03-20T17:38:12Z weight = 4 +++ ## Chapter 3.8: Databases First, we're gonna expand the configuration we defined previously. The one thing we will require is the password, although eventually I'm going to make that a command-line option using Clap. ``` rust #[derive(serde::Deserialize)] #[serde(default)] pub struct DatabaseSettings { pub username: String, pub password: String, pub host: String, pub port: u16, pub database: String, } #[derive(serde::Deserialize)] #[serde(default)] pub struct Settings { pub database: DatabaseSettings, pub port: u16, } impl Default for DatabaseSettings { fn default() -> Self { DatabaseSettings { username: "newsletter".to_string(), password: "".to_string(), host: "localhost".to_string(), port: 5432, database: "newsletter".to_string(), } } } impl Default for Settings { fn default() -> Self { Settings { port: 3001, database: DatabaseSettings::default(), } } } ``` ## Cleanup on Aisle Three The code's beginning to get a bit messsy. So let's re-arrange. I already have the configuration handler in its own file, but let's clean up the routes. I'm going to take the functions I've already defined and put them in a subfolder, `src/routes`, and then for each one I'll make a new file. For example: ``` rust use axum::{http::StatusCode, response::IntoResponse}; pub(crate) async fn health_check() -> impl IntoResponse { (StatusCode::OK, ()) } ``` Just a note: *ignore* your IDE's "advice" to remove the unneeded `async`; Axum will not compile this module correctly, `IntoResponse` requires that it be an async function. I don't know why rust-analyzer failed to pick that up. Now make a `src/routes.rs` (yes, the same name as the folder). You can activate and de-activate routes at will: ``` rust use axum::{ routing::{get, post}, Router, }; mod greet; use greet::greet; mod health_check; use health_check::health_check; mod index; use index::index; mod subscribe; use subscribe::subscribe; pub(crate) fn app() -> Router { Router::new() .route("/", get(index)) .route("/subscriptions", post(subscribe)) .route("/:name", get(greet)) .route("/health_check", get(health_check)) } ``` You don't need the `use` clauses, you could just say `index::index`, but I kinda like the way this looks, even with the repetition. It makes it clear what I'm doing and what I'm looking at. This is Rust 2021, where the presence of a file and a folder with the same name indicate a module with submodules. In prior editions of rust, instead of `routes.rs`, we would have used `routes/mod.rs`. I'm not entirely sure which I like better. Now that that's done, in our `lib.rs`, we use the `routes`: ``` rust use std::net::SocketAddr; mod configuration; use configuration::get_configuration; mod routes; use routes::app; pub async fn run() { let configuration = get_configuration().unwrap(); let addr = SocketAddr::from(([127, 0, 0, 1], configuration.port)); tracing::info!("listening on {}", addr); axum::Server::bind(&addr) .serve(app().into_make_service()) .await .unwrap() } ``` All of the Axum imports have been removed, since they're part of the routes. Nice, clean and clear code. At this point, even `cargo clippy` is happy. ### The Database String Connecting to a database these days involves a URL. Let's make it easy to generate one. Now, we could implement the trait `std::string::ToString`, or `Display`, but that would conflict with any debugging information. Instead, let's define a new method in `configuration.rs`: ``` rust impl DatabaseSettings { pub fn url(&self) -> String { if self.password.len() == 0 { format!( "postgres://{}@{}:{}/{}", self.username, self.host, self.port, self.database ) } else { format!( "postgres://{}:{}@{}:{}/{}", self.username, self.password, self.host, self.port, self.database ) } } } ``` It is possible (unwise, but possible) to connect to a Postgres database with no password. In a dynamic language this would have been less wordy, but I can't complain about Rust being a little extra verbose in exchange for being a lot more precise. ### Providing the database to the application Since we're using Postgres, in `lib.rs` we'll add `use sqlx::postgres::PgPoolOptions;`, which will allow us to create a pool of workers and limit how many there are. In the `run()` function, we're going to extract the `app()` definition and use it: ``` rust use axum::Extension; use sqlx::postgres::PgPoolOptions; // in: fn run() let pool = PgPoolOptions::new() .max_connections(50) .connect(&configuration.database.url()) .await .expect("could not connect to database_url"); let routes = app().layer(Extension(pool)); tracing::info!("listening on {}", addr); axum::Server::bind(&addr) .serve(routes.into_make_service()) .await .unwrap() ``` Now we have a configuration that is runnable. If you run it as-is, it will fail because, if you set up your Postgres database correctly, there is no password. Create a `ztp.config.yaml` file (or JSON, or TOML, or whatever floats your boat) and establish the password: ``` yaml database: password: redacted ``` At one point, I had this file named only `ztp.config`, and unfortunately it crashed. Adding `#[derive(Debug)]` to my configuration structs helped show that the password wasn't getting set, but it took me a few minutes to realize that I'd forgotten the extension so `Config` didn't know how to read the file. Naming it `ztp.config.yaml` fixed it, and this illustrates just how useful TOML can be in circumstances like this. ### Pulling in a subscription This turned out to be a bit of a beast, and I'm incredibly indebted to Carlos Armando Marcano Vargas and [his blog](https://github.com/carlosm27/blog) where he documented much of what I needed to make this work with Axum. In the migrations file, we specified that the primary key of our subscriptions table would be a UUID, and that we were going to record when the subscription happened. To do this, first we have to make sure that we have the `Uuid` and `Chrono` crates, and that they're all hooked up for proper serialization: ``` toml chrono = { version = "0.4.24", features = ["serde"] } uuid = { version = "1.3.0", features = ["v4", "serde"] } sqlx = { version = "0.6.3", features = ["runtime-tokio-native-tls", "macros", "postgres", "uuid", "chrono"] } ``` Both of the new creates need `serde` as a feature and we'll be using `UUID4` for our primary key. We'll also need a full subscription object to store, and then a way to generate it from a new subscription: ``` #[derive(serde::Deserialize)] pub(crate) struct NewSubscription { pub email: String, pub name: String, } #[derive(serde::Deserialize, serde::Serialize, sqlx::FromRow)] struct Subscription { pub id: Uuid, pub email: String, pub name: String, pub subscribed_at: DateTime, } impl From<&NewSubscription> for Subscription { fn from(s: &NewSubscription) -> Self { Subscription { id: Uuid::new_v4(), email: s.email.clone(), name: s.name.clone(), subscribed_at: Utc::now(), } } } ``` I've renamed "Subscription" to "NewSubscription" to distinguish it from the thing we're going to reliably keep in our database, which has all the fields. And now we're going to re-write the `subscribe()` function to actually talk to the database. All of those parens around the `payload` are necessary to help Rust understand what we want to borrow. You'll note that it's `payload.0`, not just `payload`; Form objects can have multiple instances of their content. ``` pub(crate) async fn subscribe(Extension(pool): Extension, payload: Form) -> impl IntoResponse { let sql = "INSERT INTO subscriptions (id, email, name, subscribed_at) VALUES ($1, $2, $3, $4);".to_string(); let subscription: Subscription = (&(payload.0)).into(); let _ = sqlx::query(&sql) .bind(subscription.id) .bind(subscription.email) .bind(subscription.name) .bind(subscription.subscribed_at) .execute(&pool) .await .expect("Failed for some reason?"); (StatusCode::OK, ()) } ``` Alternatively, I could have written: ``` rust let subscription = Subscription::from(&(payload.0)); ``` Either way works. ### Better error handling. That unused `.expect()` has Tokio panicking if I give it a duplicate email address, as one of the constraints in our original migration reads: ``` sql email text not null unique, ``` We want to catch and handle that error. We also want to fix something that's been sitting in the "TODO" list for awhile: make sure that when the subscription form isn't fully filled out, we return a straight 400. That's actually the easier part: we make the payload optional and return our default `BAD_REQUEST` if it's not there: ``` pub(crate) async fn subscribe( Extension(pool): Extension, payload: Option>, ) -> impl IntoResponse { if let Some(payload) = payload { ... (StatusCode::OK, ()) } else { (StatusCode::BAD_REQUEST, ()) } ``` So, handling the errors is going to be interesting. Going above and beyond, it's time to use `thiserror`.