Building My Own Newspaper
Writeup on my personal newspaper that I made in Rust
I really like reading independent blogs, they can range from in depth engineering blogs obsessing over bytecode to posts that just ramble about coffee to opinion pieces on space. If you're taking the time to manage a website and put content on it, I'm likely interested in what you have to say.
One major driver for me developing my own tool is that on microblogging platforms like Twitter, Mastodon or BlueSky I find that it is impossible to curate a feed of high quality content with no toxicity. Even people at the absolute top of their craft that post high quality thoughtful content can't help themselves but feed into political turmoil or argue about something. I find that negative content rarely makes it onto blogs which largely removes this issue for this project.
On BlueSky I've gone out of my way not to interact with anything political, and I've been very aggressive with blocking words but it still shows me political content pretty heavily.
My own Algorithm
When I first set out on this project the idea was to build out my own algorithm. Mastodon having zero algorithm pretty much kills it for me, I find that one person usually dominates my feed, and the overall presentation of threads is lacking. Twitter is the opposite and only shows what it thinks I want and rarely surfaces posts from my following, this is awesome for when I want to rot my brain but makes it difficult to find community.
Unsurprisingly, building an algorithm is a pretty large understaking, especially because I wanted the site to be statically generated in CI with no backend which would mean every morning I would have to get and scan every piece of content on every blog to build out an index. Then theres the issues of what makes one blog post more relevant to me than others without having any feedback since the site is static.
The "algorithm" I landed on is quite simple. I have a featured feed list and a normal feed list. The featured list is for blogs like https://ciechanow.ski where I literally put my life on pause when they post so I can read it in full. I just use the posted date for ordering posts so I add like a week to featured posts to keep them at the top. Then I remove duplicates based on root domain so that if someone is on a posting spree, or accidentally sets all their posts to the same date that they don't dominate my feed.
At the bottom of the page I pick 100 random posts from the entire pool that I collected from. This is fun because it surfaces older things that feeds I follow post.
RSS is Inconsistent
Working around the quirks of RSS is nearly half the battle of a project like this. I used Rust which was fun, but I'm sure Python has some better libraries for working with inconsistent data like this.
In the end this project is posted publicly, but its 100% just for me to consume. Its public because why not, and hopefully it gives blogs I like some small SEO boost. The HTML artifacts and occasional bad link aren't big enough issues for me to care about.
Check out the project
Also shout out to the Orion Browser for making it easy to find RSS feeds on websites, because sometimes its really really hard.
