# orderedItems[].object.content
I added the ability to manage property values on users' profiles to #Enigmatick. As I'm using this more as my full-time #Fediverse platform, I'm running into things that I've sort of blown off in the past as silly or unnecessary but finding useful now.

In this case, I implemented it using Add and Remove messages posted by the client (a single-page application in Svelte mediated by a wasm module) to the user's outbox on the server. The Actor is the target and the PropertyValue is the Object.

It's not canonically how those #ActivityPub messages should be used (e.g., managing a Collection - the attachment field on an Actor is not really a Collection). But I think logically it makes sense. And it's client communication only; those activities are not federated. When a profile is updated via the Add or Remove, a separate Update is sent out to known servers to update the profile.

Also, I may have gone a little overboard with my own properties.

# orderedItems[].object.source.content
I added the ability to manage property values on users' profiles to #Enigmatick. As I'm using this more as my full-time #Fediverse platform, I'm running into things that I've sort of blown off in the past as silly or unnecessary but finding useful now.

In this case, I implemented it using `Add` and `Remove` messages posted by the client (a single-page application in Svelte mediated by a `wasm` module) to the user's `outbox` on the server. The `Actor` is the target and the `PropertyValue` is the `Object`.

It's not canonically how those #ActivityPub messages should be used (e.g., managing a `Collection` - the `attachment` field on an `Actor` is not really a `Collection`). But I think logically it makes sense. And it's client communication only; those activities are not federated. When a profile is updated via the `Add` or `Remove`, a separate `Update` is sent out to known servers to update the profile.

Also, I may have gone a little overboard with my own properties.

# orderedItems[].object.content
Occasionally a comment to a post is deleted. Or sometimes it hasn't reached my server for whatever reason. In this case, I can tell based on the next comment that it's from Bluesky and I really just don't care enough about that service to figure out why my server may not be pulling it down correctly.

But it sucks to have broken threads because one bit goes missing. So I added in this "Unavailable Comment" component just now.

I think that will help.

#Enigmatick #Threading

# orderedItems[].object.content
Because #Enigmatick is primarily built for me, there are a lot of niceties that I haven't gotten around to adding because "I can just look in the database." I also intentionally don't have any sort of notification mechanism (I may yield on that at some point). I don't even have a field for "email" in my database Actor model.

One of those was a popover UI for Likes and Reposts. I had included the data about who liked and reposted a post months ago in the JSON that the server provides to the client, but I hadn't followed through with utilizing it beyond a basic count.

This is much better than querying my database.

# orderedItems[].object.content
I recently added direct TLS certificate management in #Enigmatick using tls-alpn-01, so all that's needed to get TLS up and running is a public IP address and a domain name pointed at that address. This is managed by a separate reverse proxy process that's bundled in the main Enigmatick binary.

I think while I'm at #DEFCON, I'm going to experiment with Arti (or anything similar) to build a Tor reverse proxy as well. I can't do automatic TLS certificate provisioning for onion services from #LetsEncrypt, but that may be okay.

# orderedItems[].object.content
Late night progress. #LateNightDevs #ActivityPub #Enigmatick

# orderedItems[].object.content
The changes I've made to #Enigmatick over the last couple of months are significant and I plan to write more about them soon. But one of my favorite is this small bit (supported by some larger structures) in my #ActivityPub crate.

impl ApCollection {
    /// Creates an async stream over all items in this collection and its linked pages.
    pub fn stream_all<F, Fut>(self, fetcher: F) -> ApCollectionStream
    where
        F: Fn(&str) -> Fut + Send + Sync + 'static,
        Fut: Future<Output = Result<ApCollection, Box<dyn std::error::Error + Send + Sync>>>
            + Send
            + 'static,
    {
        ApCollectionStream::new(self, fetcher)
    }
}

I wanted to be able to (using async code) take an ActivityPub Collection and iterate over all of its members without worrying about pulling in each page. This implementation accomplishes that.

The full source is here.

# orderedItems[].object.content
For a long time, I was trying to make the entry of text in my #Enigmatick Compose component behave as rich text like I've seen in solutions like Notion or Confluence. But I don't think I'm a strong enough front-end coder to make that work; it was always buggy and problematic.

fn main() {
    println!("Here's my new approach: a live representation of the text displayed as readers will see it");
}

# orderedItems[].object.content
To save myself some hassle, I've taken to adding functions I want in a hurry to the server command itself.

/srv/enigmatick# ./enigmatick
A federated communication platform server

Usage: enigmatick <COMMAND>

Commands:
  init         Initialize folder structure for media files
  template     Generate .env.template file
  migrate      Run database migrations
  cache        Manage cached media files
  system-user  Create or ensure system user exists
  server       Start the web server and background tasks
  instances    Manage federated instances
  send         Send various activities
  muted-terms  Manage user muted terms
  help         Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version


/srv/enigmatick# ./enigmatick cache
Manage cached media files

Usage: enigmatick cache <COMMAND>

Commands:
  prune          Prune cached files older than specified duration (e.g., 30d, 2m, 1y)
  delete         Delete cached item by URL
  delete-server  Delete cached items from server/domain pattern
  help           Print this message or the help of the given subcommand(s)

Options:

  -h, --help  Print help

/srv/enigmatick# ./enigmatick send
Send various activities

Usage: enigmatick send <COMMAND>

Commands:
  actor-update  Send actor update to all known instances
  help          Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help

The commands here run with direct database access. But my plan is to extend a lot of this (and much more) as a network-based CLI tool (e.g., maybe as ek in the enigmatick_wasm crate).

#Enigmatick #RustLang #ActivityPub

# orderedItems[].object.content
I've built #Enigmatick to manage threading using conversation IDs like Mastodon uses. But that's not aligned with #ActivityPub - or at least it wasn't the least time I looked. ActivityPub uses inReplyTo to indicate direct parent relationships and doesn't have a concept for linking multiple objects under a single ID.

Other systems don't use the conversation convention and that's caused some discord in my timeline. So now I'm learning about closure tables and concepts I probably would already know if I had studied computer science in college. But it's never too late to learn!

# orderedItems[].object.content
I reworked my image processing on #Enigmatick recently but haven't had a chance to try it out. Here's a picture of my son checking in with the neighbor llamas a few years ago.

# orderedItems[].object.content
Migrated my followers from #Serendipitous to #Enigmatick. The process was pleasantly simple - even though I haven't implemented Move activities on the new system yet. I'll tackle that shortly as I see I have a backlog of those in my unprocessable queue.

It also seems like I need to expand my database connection pool; all of those Follow activities coming in at once was a little messy.

# orderedItems[].object.content
Using my amateur woodworking skills, I built myself a server rack using red oak and 72" metal rails last year. It lives in a dedicated corner of my basement where sound and heat are manageable challenges. It has since housed several enterprise switches (SONiC) and routers (VPP) I've been building around.

My venerable DS1513 NAS turned is now 12 years old, so I decided to leverage the rack to move to a Supermicro X11-based server running TrueNAS SCALE to replace it. 12 drives, 40 cores, 2 x 25Gb NICs, and 128GB of ECC RAM later, I may have over done it a little. But it's a very nice upgrade. I finished moving the NFS shares for this service over tonight and migrating to a new database run on the NAS server itself (which eliminates a long-standing pain point with iSCSI).

I'm considering just turning down my long-running Mastodon instance in favor of leaning in to #Enigmatick. As a result, I'm sending out lots of follow requests. Feel free to follow me back here, as this is what I'll be paying attention to moving forward.

# orderedItems[].object.content
#Enigmatick's timeline retrieval performance was getting a little slow, so I spent some time optimizing my PostgreSQL queries yesterday. I was able to take the EXPLAIN ANALYZE loop from 17 seconds (an exaggeration of the actual experience in production of a delay of about 2 seconds) to 53ms. It was mostly down to a JOIN that employs a JSONB_AGG to pull in Actor records associated with an Activity to avoid having to make additional calls. The way I was matching against the attributed_to JSONB column was apparently slowing things down.

I also have some #Wasm calls that proactively transform #E2EE asymmetrically encrypted messages to symmetrically encrypted vault items. Those run on each load of the timeline, but I was able to use spawn_local to move the processing to a background task for timeline views that don't display encrypted content.

# orderedItems[].object.content
Integrating the shared jdt_activity_pub #RustLang crate used by the backend #Enigmatick server into the frontend #Wasm component. Necessarily broke a couple of Vodozemac functions, but those will be replaced by #OpenMLS shortly.

# orderedItems[].object.content
One interesting thing I discovered about the RsaSignature2017 implementation in Mastodon is that rather than combining the options and message hashes as their native binary representation like: let combined = [options, message].concat() to use in the signature verification like verifying_key.verify(&combined, &signature), Mastodon converts the hashes to a hex string, smashes the textual representation together, and then performs the signature on the bytes of that (which means that we have to do the same in reverse for verification).

let options_hex = hex::encode(options);
let result_hex = hex::encode(result);
let combined_hex = format!("{options_hex}{result_hex}");
verifying_key.verify(combined_hex.as_bytes(), &signature);

It's inefficient; I suspect #Ruby may make operating on the bytes themselves challenging. But I'm just guessing, since it's been quite a while (10+ years) since I've done much in that ecosystem.

It'll be nice when the #Fediverse evolves past this bit of history.

#RustLang #Enigmatick #ActivityPub #RsaSignature2017

# orderedItems[].object.content
I took the time this evening to wade through the swamp of sadness that is #JsonLD and the idiosyncrasies of how it's implemented in Mastodon. However dubious the whole effort may be, I now have a working prototype that I'll add to #Enigmatick in the next few days.

My motivation is to allow me to connect my server to a relay with some confidence that I can verify the source of the messages that are being relayed in. There are newer standards that I want to build to eventually. But the reality is that this bastardized #RsaSignature2017 format is what's in use for the vast majority of messages today.

I'll add the routines to my #jdtactivitypub crate (on crates.io) so that other #RustLang folks have a working example. I'm also going to try to add my HTTP signature and verification methods to that crate.

# orderedItems[].object.content
I've become increasingly weary of the task of keeping my backend and #WASM structs aligned. So I decided to break #Enigmatick's #ActivityPub structs out into their own crate that I can use in both places. I have the Inbox and Outbox traits moved over into my core crate and just need to complete the removal of any dependencies on the data model structs.

Maybe this AP-focused crate can help other #Rust #Fediverse devs eventually.

# orderedItems[].object.content
I write posts here on #Enigmatick using #markdown. The web interface integrates the Showdown converter combined with highlight.js.

As such, my posts often have elements that might not look great on Mastodon and other platforms that strip a lot of the context out. They seem to do an okay job of making code blocks look different, but in a pretty terrible way.

If you want to see what I intend the posts to look like, use the "Open Original Page" on any of my posts in Mastodon to open the URL directly to my web server.

# orderedItems[].object.content
#Enigmatick has three sending modes today: Public, Direct, and Private. I've been working on tweaking the Public mode this morning. It was built so that it would compile inboxes from the list of a sender's followers, while deferring to the sharedInbox attribute where it exists. I've altered that so that Public posts are now sent to the sharedInbox of all instances that have communicated with my server in the last two weeks and any individual inboxes for followers on systems that do not use a sharedInbox. This is one of the queries that facilitates that change.

pub async fn get_instance_inboxes(conn: &Db) -> Vec<ApAddress> {
    let cutoff = Utc::now().naive_utc() - chrono::Duration::days(14);

    conn.run(move |c| {
        instances::table
            .filter(instances::blocked.eq(false))
            .filter(instances::shared_inbox.is_not_null())
            .filter(instances::last_message_at.gt(cutoff))
            .select(instances::shared_inbox.assume_not_null())
            .get_results::<String>(c)
    })
    .await
    .unwrap_or_default()
    .into_iter()
    .map(ApAddress::from)
    .collect()
}

I also made a significant change to the dispatching of these events, leveraging tokio to avoid having to wait for each connection to complete sequentially.

let handles: Vec<JoinHandle<LogMessage>> = inboxes
        .into_iter()
        .map(|inbox| {
            let client = client.clone();
            let body = body.clone();
            let profile = profile.clone();

            tokio::spawn(process_inbox(inbox, body, profile, client))
        })
        .collect();

    let mut logs = Vec::new();
    for handle in handles {
        if let Ok(log) = handle.await {
            logs.push(log);
        }
    }

I'm planning to add a fourth mode for Group Private #E2EE messages (i.e., encrypted for multiple group participants). I have the concept in my head; I just need to sit down and build it.

#ActivityPub #RustLang #WASM

# orderedItems[].object.content
I redesigned the sending function in #Enigmatick to use tokio::spawn. This message is the first test.