From 2bd7c0aaadc5b08bab6a2335a89cae1c33ebbbfe Mon Sep 17 00:00:00 2001 From: "Elf M. Sternberg" Date: Sun, 27 Sep 2020 05:35:37 -0700 Subject: [PATCH] DOC Phase one committed to the project, the basic layout of the backstore. Just describing my vague TPP-level commitments to the project. --- README.md | 16 ++++ docs/00_Basic_Ideas.md | 153 ++++++++++++++++++++++++++++++++ server/nm-store-cli/.gitignore | 1 + server/nm-store-cli/Cargo.toml | 9 ++ server/nm-store-cli/README.org | 19 ++++ server/nm-store-cli/src/main.rs | 3 + server/nm-store/.gitignore | 2 + server/nm-store/Cargo.toml | 16 ++++ server/nm-store/README.org | 27 ++++++ server/nm-store/src/lib.rs | 7 ++ 10 files changed, 253 insertions(+) create mode 100644 README.md create mode 100644 docs/00_Basic_Ideas.md create mode 100644 server/nm-store-cli/.gitignore create mode 100644 server/nm-store-cli/Cargo.toml create mode 100644 server/nm-store-cli/README.org create mode 100644 server/nm-store-cli/src/main.rs create mode 100644 server/nm-store/.gitignore create mode 100644 server/nm-store/Cargo.toml create mode 100644 server/nm-store/README.org create mode 100644 server/nm-store/src/lib.rs diff --git a/README.md b/README.md new file mode 100644 index 0000000..750a2ad --- /dev/null +++ b/README.md @@ -0,0 +1,16 @@ +# Notesmachine + +> A Second Brain, From First Principles + +--- + +Notesmachine is a collection of tools for organizing your notes around +first principles. It has graphical desktop tools, a web-based +interface, and mobile tools for collecting, organizing, and +re-discovering the knowledge that you personally collect. Notesmachine +is primarily a [Zettlekasten](https://zettelkasten.de/posts/overview/) +engine designed to work on a personal scale, but it is also designed to +make each note smarter, more capable, and more informative, as well as +making capturing notes not only possible, but simple and direct. + + diff --git a/docs/00_Basic_Ideas.md b/docs/00_Basic_Ideas.md new file mode 100644 index 0000000..be12c2c --- /dev/null +++ b/docs/00_Basic_Ideas.md @@ -0,0 +1,153 @@ +# Basic Ideas of Notesmachine + +## Zettlekasten + +Zettlekasten is a note organization method to facilitate +the production of intellectual work. You have a collection of boxes +into which notes from different *areas* of your life belong. When you +have assembled enough notes to compile a whole work out of them, you +take them out of the box and arrange them on a table, re-working them +if necessary, until you have the rough draft of a whole work. The +notes are organized into a *hierarchy* of knowledge about that area, +and when you have an idea, you have one of three steps. +- If your collection is small to non-existent, you can just add the + note, annotating it in a way that tells you where it goes. +- If your collection is large enough you can't be sure where it goes, + look to see if a similar note is already present. If it is, you + have two choices: + - If the idea very similar, just write your idea into the existing + note. + - If the idea is "kinda" similar, write a new note that compiles the + idea into a single idea + - If the idea is new, write a new note, and mark it in ways that + place it near its relatives. + +What those boxes are is up to you. For example, I love to cook, write +software, and write stories. It is *unlikely* that there's much overlap +between those three categories, so I have separate boxes for each of them. + +I said it is unlikely that there's overlap, but it's not impossible. +For example, I like to blog about all three of those things, and so +having things tagged somehow as "blog idea" would be useful, rather than +having a separate box for "blogging." More to the point, Notesmachine +itself is a powerful tool for *writing stories*. + +Notesmachine is *primarily* a Zettlekasten support tool, but as we'll +see, combined with a computer's ability to reach across itself and find +other notes, it can produce powerful knowledge-building abilities. + +## The Noguchi Method + +The Noguchi Method is more project-oriented. It still uses a +boxes-and-notes method, but it's more focused on the projects. Each +project has its own clearly labeled box, and the boxes are arranged in a +straight line on a shelf. (Noguchi himself uses those narrow, +square-bottomed plastic folders you can find at office supply stores.) +Whenever he *adds* something to a folder, the folder is moved to the +*leftmost* position of the shelf. Folders he *refers to* but does not +*add to* stay where they are. Over time, the *rightmost* folders become +irrelevant to ongoing projects, and on a fairly regular basis he +archives those folders, making room for new projects. + +By marking every box in Notesmachine by when it was *created*, +*updated*, *requested*, and finally *deleted*, it becomes possible to +create a Noguchi-like view of the boxes you care about. + +## Bisecting K-Means + +This isn't a method, it's an algorithm by which documents, usually quite +large documents, are "clustered" into groups based on the prevalence of +certain terms within the documents, as is used as automated way of +identifying groups of documents. + +Notesmachine *may* provide a few different variants on the K-Means +algorithms in the future, but these algorithms are known to be expensive +to run and difficult to make progressive, but it might prove incredibly +powerful in the future to show that unrelated notes have some overlap +based on their content. + +## Keyword discovery + +Roam provides a powerful tool in the form of "keyword discovery." Roam +scans new notes for terms that already appear regularly in its list of +keywords, and asks the user if it wants those terms to be made into +links to the boxes that represent those keywords. Notesmachine provides +this, but only as a separate *feature*. + +## Note cloning + +Notes may be related to other notes, but sometimes its not enough to say +that one note is related to another. Sometimes you want the note *in +two boxes*, and in that case it's enough to copy the note. Sometimes +you want the note *in both boxes simultaneously,* such that editing the +note in one place makes those edits to all the notes in all other boxes. +Notesmachine provides the ability to *duplicate* notes, have notes +*refer to* other notes, or *fully embed* notes in multiple locations in +your box collection. + +## Box merging + +Sometimes you may find yourself with multiple boxes with the same kinds +of notes in them. I have a box "Recipes" and another "Entrees". If I +want to merge those two boxes, all I have to do is tell Notesmachine +that I want all "entree" tags changed to "recipe". + +Notesmachine's links engine uses *three different* algorithms when +labeling a box: The first is that it has a "title" for the box; the +second is that it has a "slug" for the box, which removes stopwords and +produces a simplified url-ready all-lower-case form of the title; and +the third is that it *stems* the title, which uses the Porter Stemming +algorithm to try and find *related* boxes such that "entrees" and +"entree" will be treated the same. The preferred Porter variant is the +one provided by +[NLTK](https://www.nltk.org/api/nltk.stem.html?highlight=porter%20stemmer). + +The dataset used for stopwords is the +[spark](https://github.com/loony-bean/stopwords-rs/blob/master/src/spark/data/english.txt) +collection. + +## 101% Markdown + +There are a number of different means by which notes can be entered, but +the two fundamental ones are simply as a stream of text-- which can be +useful when using voice input-- and Markdown, which provides *some* +structure and is generally more useful. Notesmachine recognizes two +different means of tagging: + +- [[classic]], where the tag is surrounded by double-brackets. In this + mode, +- #Hashtag, where the tag is prefixed by a cuddled hash symbol. (A hash + symbol in the first column followed by a space is a *title*; anywhere + else, it's just a hash symbol.) This also has two other features: + - (feature) `#snake_case_hashes` will automatically be interpreted as + separate words. + - (feature) `#CamelCaseHashes` will also be automatically be interpreted + as separate words. + - (feature) `#lisp-case-hashes` will likewise be treated as separate + words. + +## Three kinds of relationships: + +Notes machine supports three different kinds of relationships. + +- **Links**: Every note in a box can be linked to other boxes. That's + sorta the idea of NotesMachine, that you can have lots of notes and + they all relate to one another somehow, creating a web of notes that + on their own can collect ideas and spark new ones. Along with the + most common sort, one note links to another box, notes can link to + specific notes in another box, and then there's referring and + embedding. +- **Nests**: Notes can be nested *inside other notes*. This is a common + organizational scheme, and it results in the sort of structure of + folders-within-folders that's common in most operating systems. Nests + are useful because they automagically become *outlines* for documents + and projects. +- **Boards**: Every once in a while it's nice to take all the notes out + of a box and arrange them physically, visually in front of you. It + would be nice if we could support that as a display form. + + + + + + diff --git a/server/nm-store-cli/.gitignore b/server/nm-store-cli/.gitignore new file mode 100644 index 0000000..ea8c4bf --- /dev/null +++ b/server/nm-store-cli/.gitignore @@ -0,0 +1 @@ +/target diff --git a/server/nm-store-cli/Cargo.toml b/server/nm-store-cli/Cargo.toml new file mode 100644 index 0000000..2a654e5 --- /dev/null +++ b/server/nm-store-cli/Cargo.toml @@ -0,0 +1,9 @@ +[package] +name = "nm-store-cli" +version = "0.1.0" +authors = ["Elf M. Sternberg "] +edition = "2018" + +# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html + +[dependencies] diff --git a/server/nm-store-cli/README.org b/server/nm-store-cli/README.org new file mode 100644 index 0000000..bbb197c --- /dev/null +++ b/server/nm-store-cli/README.org @@ -0,0 +1,19 @@ +* Notesmachine Store CLI + +This is the storage layer's CLI. While the CLI will never provide +access to the internal IDs of whatever store is available, it will allow +users to initialize (and destroy) the database (provided permissions are +set correctly), and read and write atomic entities from the database for +analysis and approval. + +** Plans +*** TODO List notes +*** TODO Write a note +*** TODO List notes again +*** TODO Retrieve one note +*** TODO Write a second note +*** TODO Associate the two notes in a parent/child relationship +*** TODO List pages +*** TODO Save a page +*** TODO Connect a note to a page +*** TODO Get a page and all its notes (CTE time) diff --git a/server/nm-store-cli/src/main.rs b/server/nm-store-cli/src/main.rs new file mode 100644 index 0000000..e7a11a9 --- /dev/null +++ b/server/nm-store-cli/src/main.rs @@ -0,0 +1,3 @@ +fn main() { + println!("Hello, world!"); +} diff --git a/server/nm-store/.gitignore b/server/nm-store/.gitignore new file mode 100644 index 0000000..96ef6c0 --- /dev/null +++ b/server/nm-store/.gitignore @@ -0,0 +1,2 @@ +/target +Cargo.lock diff --git a/server/nm-store/Cargo.toml b/server/nm-store/Cargo.toml new file mode 100644 index 0000000..3326a38 --- /dev/null +++ b/server/nm-store/Cargo.toml @@ -0,0 +1,16 @@ +[package] +name = "nm-store" +version = "0.1.0" +authors = ["Elf M. Sternberg "] +edition = "2018" +description = "Datastore interface layer for notesmachine." +readme = "./README.org" + +[features] +cli = ["nm-store-cli"] + + +[dependencies] +nm-store-cli = { path: "../nm-store-cli" } + + diff --git a/server/nm-store/README.org b/server/nm-store/README.org new file mode 100644 index 0000000..63b2037 --- /dev/null +++ b/server/nm-store/README.org @@ -0,0 +1,27 @@ +* Notesmachine Store + +This is the storage layer for Notesmachine. It supports the retrieval +of the basic information from Notesmachine, but it never reveals the +underlying mechanisms by which it performs its magic. The "ids" leaving +this API will always be the textual ids used in all public +representations: + +- Boxes are referenced by slugs, and clients must accept uniqueness + failures. + +- Notes are referenced by "friendly_id" format strings, which are + base-62 formatted versions of UUID4 strings. + +** Plans + +*** TODO Make it possible to save a note +*** TODO Make it possible to retrieve a note +*** TODO Read how others use SQLX to initialize the database +*** TODO Implement CLI features +*** TODO Make it possible to connect two notes +*** TODO Make it possible to save a page +*** TODO Make it possible to connect a note to a page +*** TODO Make it possible to retrieve a collection of notes +*** TODO Make it possible to retrieve a page + + diff --git a/server/nm-store/src/lib.rs b/server/nm-store/src/lib.rs new file mode 100644 index 0000000..31e1bb2 --- /dev/null +++ b/server/nm-store/src/lib.rs @@ -0,0 +1,7 @@ +#[cfg(test)] +mod tests { + #[test] + fn it_works() { + assert_eq!(2 + 2, 4); + } +}