BackSlasher

Audible MP3 Downloader

2025-12-01T00:00:00+02:00

I have a friend that is an avid Audible user. They have a big library and enjoy the service very much. They are also an avid swimmer, and have a water-proof media player they can listen to when swimming.
They would really like to listen to legally-purchased audiobooks in the pool, but found no proper device that can both do Audible and survive underwater.

My first suggestion was using some Bluetooth trickery, but it seems that it doesn’t really work underwater. A bit of research showed that underwater radio is not a solved problem, so I left this research venue alone. Last weekend, using some other people’s work and Python-glue, I got them a website they can:

Log in to Audible
See their library
Choose books to download-convert
Get a converted book as a zip

Some nitty-gritty

FastAPI, uv for dependency management.
Vibe-coded with Claude CLI.
Uses the audible and audible-cli Python packages for Audible interaction.
Comes with a Docker image, but can run without (if you have a proper ffmpeg setup)
Has an SQLite db for saving Audible connections (per browser session) and job management, it’s ok not to persist it.

Get it while it’s hot

https://github.com/BackSlasher/audible-downloader

Ship Today, Scale Tomorrow #1: Different Servers for Different Jobs

2025-11-05T00:00:00+02:00

Working with early-stage CTOs, I see two traps repeatedly: over-engineering that steals time today, or decisions that choke growth tomorrow. I wanted to share lessons from the field.

“We already have a Python server for the API, so we figured we’d let it do the AI model training too.”

This explained why their API server had grown from a tiny VM into three always-on GPU servers. API requests failed competing with training jobs. Training runs failed competing with each other. All to handle periodic bursts of heavy workloads.

People can think “this is the server we have” - you already provisioned this server, so it’s easy to add more work to it. But one of the great things about the cloud is you can spin up different machines for different purposes, and only pay for what you use.

We moved training to a batch system that spun up GPU machines only when needed, and as many as needed. The API dropped back to one tiny server. No conflicts, lower costs, reliable performance.

Back-office work (batch jobs, training, reports) comes in bursts and tolerates delays. Front-office work (like user-facing APIs) needs speed and consistency. When they mix, they interfere with each other. When you’re small, one server is probably fine. When you start feeling resource contention, it’s time to split.

Getting a Git remote from a Sapling repo

2025-11-02T00:00:00+02:00

Assuming you have a git-backed sapling repo (e.g. you used sl clone on a git repo, or sl init --git) and you want to access the commits from a git repo, here is the git remote incantation:

git remote add local $PATH_TO_SAPLING_REPO/.sl/store/git

Flashing ESPHome with Docker

2025-10-20T00:00:00+03:00

ESPHome’s WebSerial connection didn’t work for me, and the computer running ESPHome was a bit far.
Instead, I decided to run a local copy of ESPHome to flash the firmware over USB, and figured it’s an incantation worth keeping

First, save your target yaml locally, e.g. device.yaml.
Then:

docker run --rm -it --device /dev/ttyUSB0 -v .:/app:ro ghcr.io/esphome/esphome run /app/device.yaml --device /dev/ttyUSB0

Some breakdowns:

docker run --rm -it is the usual. Remove image when done, use tty
--device /dev/ttyUSB0: Mount ttyUSB0 (your esp device) to the docker container
-v .:/app:ro make the local directory available as readonly to the container as /app
image name
run /app/device.yaml flash said file, which we mounted
--device /dev/ttyUSB0 flash to this device

Keep this running until you see the device log reporting successfully connecting to wifi

Tapestry

2025-10-18T00:00:00+03:00

This is a big writeup of a project I’ve been ideating for a couple of years, and working heavily on in the last couple of months.
It involves Python, hardware, 3D printing, a bit of computer vision, and a surprising amount of vibe coding.

All code and 3d models are available in:

What and why

TL;DR - a big display, composed of multiple smaller electronic paper displays.

The long way - ever since I first owned a Kindle, I was impressed by the unique quality of images whenever the Kindle was on screensaver mode, and figured how great it’d be to have a full-sized poster with this unique quality. Since full-sized e-paper displays are crazy expensive (thousands of USD) and probably use some terrible software instead of an API, I figured I can try building one from multiple smaller displays.

History

I started with a rpi and Waveshare screen, specifically IT8951. While I managed to get it working by copying the code and connecting the hat, I couldn’t get it to work using USB / SPI, meaning I needed 1 rpi per screen, which seems expensive and complicated.
When connecting the USB interface, it showed up as a block device, and the demo software they provided (binary, no source) worked by sending specific weird instructions to the block device.
I couldn’t get their windows-based demo to work, and disassembling it didn’t make me any wiser. Asking their support for help, they said they’re unable to provide source code, so I was stuck.

When idly browsing the internet, I found EPDiy, which is an ESP32-based FOSS solution (hardware+firmware) for controlling e-paper devices. The author had some demos that looked quite promising, and was really friendly when we chatted. I decided to try and create a solution based on that.

PoCing the idea involved setting up multiple screens with multiple EPDiys, having them join my home network and driving them from some Python code from my laptop.
Power was from my Framework laptop’s power brick, with multiple dumb USB-C splitters.
The resulting setup produces great images, but was very delicate. Many screens were destroyed during experimentation, especially their delicate ribbons. The recent breakthrough was with the case design, which is detailed later in this post.

Composition

rpi controller
“nodes” composed of e-paper displays, their driving custom PCBs, and a case holding them together
Board everything is mounted on
PSU and wiring

Controller

RPi4, running Arch Linux ARM.

Networking

$ nmcli con
NAME                UUID                                  TYPE      DEVICE
tapestry            afd20737-a86d-4969-a051-f349c7fa3633  wifi      wlp1s0u1 
home-wifi           2183e6b3-13bb-4da6-8e93-13dd9811433b  wifi      wlan0
lo                  87be8e66-b14e-4b99-b8a5-fe91ef924416  loopback  lo

Run by NetworkManager.
Multiple WiFi cards. Built-in card used for connecting to my home WiFi. Additional USB one (MediaTek Inc. MT7612U) used as an Access Point (“WiFi host”) using NetworkManager’s “Shared” internet connection.
The “Shared” network is internal for communicating with the nodes.
NetworkManager spawns dnsmasq for DHCP which has a leases file I utilize for positioning (see later on).

I use NM because it’s minimal-hassle and works relatively well.
I don’t need the upstream internet access the “shared” config is providing, but it doesn’t hurt so far.
What I am missing is the ability to resolve WiFi clients’ hostnames from the AP. As a workaround, I address them by IP addresses.

Controller software

$ systemctl status tapestry-webui
* tapestry-webui.service - Tapestry Web UI - Distributed E-ink Display Controller
     Loaded: loaded (/etc/systemd/system/tapestry-webui.service; enabled; preset: disabled)
     Active: active (running) since Thu 2025-10-16 22:01:58 IDT; 1 day 14h ago
 Invocation: d7d6ce0b8a1c4bbfab438208672668c0
   Main PID: 4456 (tapestry-webui)
      Tasks: 3 (limit: 4915)
        CPU: 28min 24.779s
     CGroup: /system.slice/tapestry-webui.service
             `-4456 /home/nitz/controller/.venv/bin/python3 /home/nitz/controller/.venv/bin/tapestry-webui

A Python process, spawned by a systemd service with autostart.
Env managed with uv.
Flask for “people facing” WebUI / API.
Composed of two “modules”.
The “backend” module in charge of interacting with the nodes, slicing images, converting them into the nodes’ binary format etc, was written by me a long while ago.
The “frontend” module that handles WebUI was almost completely vibe-coded. I acted more as a PM there.

Node firmware

Controller has a copy of the firmware so it can flash it into nodes.
esp-idf-helper from the Arch Aur repository used to grab a specific idf.py version I found to work.
The controller is able to git pull my node firmware from Github, then use a script to:

Initialize idf.py
Create a CSV file for populating the node’s NVS (“non-volatile storage”, key-value) with operational parameters (connected screen type, internal WiFi credentials).
Build binary including partition table, flash to device over USB

Additionally, the controller can use an HTTP endpoint on the node to send OTA changes of firmware (e.g. changes in the nodes’ HTTP server, support for more displays) or parameters (e.g. new WiFi config).

Deploying new Controller code

Done from my laptop via a makefile incantation that involves rsync, uv sync, and restarting the systemd service.

Node: PCB

Custom-made PCB made using EPDiy v7, based on ESP32-S3.
I understand very little about the hardware, and I found a service (jlcpcb.com) that will take a schema and a BoM (and some money), and mail me ready-to-be-used PCBs.
I added a prototype of my HTTP server to the EPDiy repo. However, it doesn’t include the config-reading OTA-supporting fanciness that the current iteration has.

Coding-wise, the bootstrapping and drawing logic was done by me, but a lot of the later additions (e.g. OTA) were vibe-coded with me reviewing the code thoroughly.

Boot

The software is an HTTP server, with the base logic being:

Read config from NVS
Initialize screen
Connect to WiFi. If failed, wait for 30s and try again (to handle startup races with the controller)
Start an HTTP server on port 80, handle requests as they come in.

The requests are (at the moment):

GET /: Reply with data about configured screen - temperature, resolution
POST /clear: Clear the screen
POST /draw: Draws on the screen a given custom-format (4-bit grayscale bitmap) image.
GET /ota: Data about currently running firmware version
POST /ota: Upload a new firmware binary using OTA, verify and restart
GET /ota/parameters: Get current configuration. WiFi password is redacted
POST /ota/parameters: Set new configurations

OTA

(Over the Air updates)
I chose a “safe” OTA methodology:

The device is partitioned to have 2 “application” partitions. While the bootloader is using one, the other is available to have a new version written into it.
After OTA, the bootloader switches to the new partition in “test” mode. If the new firmware won’t boot, crashes out, or doesn’t confirm it’s healthy within 5 minutes, the upgrade is considered a failure and the bootloader switches to the previous partition.

This was a bit tricky to implement, as the default application size partition is 1MB, but the device has 2MB memory, meaning there’s no room left over for bootstrap and NVS partitions if we take 2 application partitions at that size.
Some measuring led me to find out that the current binary is only ~800K, so the app partition can be 900K (a bit more because of alignment I think), and room was found for the other partitions.

Node: Display

I chose ED097TC2, a biggish screen from EPDiy’s support matrix, and ordered a bunch from AliExpress.
I experimented with smaller screens (ED060XC3), but their awkward positioning relative to the EPDiy’s connector made the nodes less aesthetically pleasing

Node: Case

The node’s components (PCB, e-paper display) are coupled together by a case that holds the display in place with three grooved “arms” that the screen slides into from the top, and on the other side has mount holes for the PCB. Both are positioned so the ribbon connecting them is on a good balance of tight/free, and protected from stress (as before the case, the ribbon tearing was the most common cause of my prototypes being destroyed).
The cases are designed by mostly by the LLM and 3D-printed by me.

Design

I don’t consider myself good at physical engineering, and for a long while designing a case was the blocker for my project. The pairing was so custom that there was nothing in the wild I could use, and I couldn’t convince my few industrial-design friends to help me on this.
Recently I figured that since “AI” is taking over everything, maybe it could take over this. I looked for some foss-autocad-like-MCP server, and found that freecad has an MCP server here.

I couldn’t get it to work.

When using Gemini CLI, it made very silly mistakes and insisted everything is OK.
Claude CLI couldn’t get the MCP server to work, and suggested it write SCAD files instead and I can compile them. This was a MAGNIFICENT idea.
We ended up with the following work procedure:

Claude generates / modifies scad files
It can use CLI tools to get renders of the model, and feed said renders to its image recognition to get some clue on how the end result looks (did not work perfectly)
I have OpenSCAD open on the same file, able to pick up changes on file writes and review the LLM’s work
OpenSCAD can compile into an STL I can print
Being text, everything can be stored in a git repo

This methodology helped me greatly. Some impressions from it:

During our work, I occasionally had to tell the LLM that the model it generated doesn’t work (e.g. pieces floating in the air), which I consider a failure, but manageable.
When I wanted to describe a situation, e.g. some screens stacking next to each other, I used ASCII symbology like so:

Imagine that this is a single screen:
```
|---|
  |
```
I want to stack them like this:
```
    |---|
|---| | |---|
  | |---| |
      |
```
Or this to describe a keyhole mount, where the bottom area is wide enough to let the pin head go through, but the top is narrow so it can’t leave:
```
 ___
 | |
 | |
/   \
|   |
\---/
```
And surprisingly enough, the LLM understood. Not only by saying it did, but by adjusting the case designs to match my requests.
My silly terminology (“keyhole mounts”, “arms” vs “tendrils”) persisted into the resulting SCAD file, meaning I could reuse it when conversing with the LLM about changes rather than having to describe the specific block I was interested in.

Printing

I use a Prusa MK3S+ printer I bought and assembled a couple of years ago. It has an RPi4 mounted on it running OctoPi and OctoPrint.
I’m using whatever PLA I have lying around, since prototyping means there’s a high turnover of the prints (e.g. I added external grooves to the cases and needed to replace all of the perfectly-working ones I previously had).
Slicing is done via PrusaSlicer (based on Slic3r).
Each case takes ~5 hours to print.

I find it incredible to think that I have a small factory in my study, ready to print stuff I design on my computer. Even more so when you consider I assembled this printer from parts that I relatively understand how they work, so there’s no magic involved, just excellent engineering.

Functionality

The webui allows the user to:

See what is currently displaying - both the source image and how it’s divided into the screens
Post a new image to be displayed (file upload or drag and drop)
Set up a “screensaver”, which can be:
1. A directory of images, random-picking from the files
2. A subreddit, random-picking from the top N posts containing photos
3. Pixabay (given an API key), random-picking from the search results of definable keywords. I stopped following through there because I noticed their search is broken and didn’t let me filter out things I don’t like
Flash over USB / OTA over WiFi nodes
Configure the displays’ layout through QR positioning

Positioning

I’m quite proud of this, so I think it’s worth its own section. The controller has a layout yaml file that looks like this:

devices:
- coordinates:
    x: 185
    y: 141
  detected_dimensions:
    height: 102
    width: 147
  host: 10.42.0.166
  rotation: 180
  screen_type: ED097TC2
- coordinates:
    x: 23
    y: 210
  detected_dimensions:
    height: 102
    width: 149
  host: 10.42.0.98
  rotation: 180

For each node, you have the host (IP over which it can be contacted), its positioning in the canvas (x,y and rotation), and the size of the node’s display. I managed to produce early versions of this file manually, but found it tedious before taking things like complex rotation (not 90° multiples) into account.

Instead, I produced a more automated process:

Since all nodes have the WiFi credentials for the internal network, we query the DHCP records of the dnsmasq server for all current DHCP clients. From it, we extract the assigned IP address (as we sadly can’t use the hostnames, as NM won’t configure the OS to consult the dnsmasq server).
We end up with a list of IP addresses of the nodes
For each node, we query it to get the screen type it has and its resolution. We then generate a personalized QR code for that node that contains:
1. The node’s IP address
2. Screen type
3. Resolution (px height, width of screen)
4. Resulting px size of the QR code (60% of the min between screen width/height, calculated at QR generation time).
  We stick all this in JSON as the QR payload.
Each node then displays its QR code
User takes a photo of the layout with all of the QR codes and uploads to the WebUI (it’s really easy from a smartphone)
We detect all QR codes in the photo (and match our payload pattern) and extract the payload + bounding box of each of them using cv2’s QRCodeDetector::detectAndDecodeMulti
Iterating over them, we extrapolate the screen’s bounding box:
1. Take the QR’s bounding box
2. Calculate center
3. Read from QR payload the size of the screen and the size of the QR code, determining the ratio between screen and QR size
4. Use the ratio to move the QR’s bounding box further away, generating the screens’ bounding box
For each screen, determine from the bounding box the size, positioning and rotation. Align them with the payload from the QR to generate an entry
Ensure that all nodes from the IP list are present in the resulting collection. If not, warn the user that some are missing

We preview the layout to the user, and assuming they accept it we write to the file and redraw the image.

This setup took quite a bit of brainwork from me (and a lot of LLM tokens, I bet), but it works pretty impressively and the resulting composite photos are pretty great.
An obvious “duh” moment was when I stopped the ridiculously long feedback loop of testing the system via the webui every time, but rather saved one of the QR-positioning photos and created a small CLI to color the bounding box (and the calculated screen one) so I can iterate much quicker.

The revelation leading to the biggest improvement of the photo you see above is due to the QR size calculation being done incorrectly. The image of the QR code was indeed taking 60% of the screen, but said image contained a white border (usually needed to protect the QR code from random stuff around it), but the border was not detected in cv2’s bounding box, which made the QR box smaller than intended. The relevant code now looks like this:

qr = qrcode.QRCode(
    version=None,
    error_correction=qrcode.constants.ERROR_CORRECT_L,
    box_size=1,
    border=0,  # Entire image is white, no border needed
)
qr.add_data(qr_data)

Mounting

All of the cases are mounted on an IKEA SKÅDIS pegboard. Most of them using the hooks, with the cases designed with mounts for the hooks.

Power

I’m currently using a “usb charging station” from AliExpress that is rated for 5v/60a (300w), which is way overkill. All of the devices (controller + nodes) are connected via USB-C cables, which are also overkill.
I’m hoping to switch to a smaller PSU (100w) and simpler electrical wiring to reduce complexity and waste. Since this is only 5V, I’m not as worried about electrocuting myself as I’d be with higher voltage.

Vibe Coding

While I’m wary of people overusing LLMs for their day job (I had several encounters with “I dunno the AI put it there” explanations in code review), this project would have been stuck without the power multiplication LLMs give people who sort-of-know what they’re doing. Although clueless in 3D design, I was able to produce cases that match my use case perfectly, and even though I’m not very strong in web stuff, the WebUI for Tapestry is extremely elegant.

My setup started with Gemini CLI, but I moved to Claude CLI as per my friends’ recommendation. No agents or MCP servers, no memory and barely any README.md or CLAUDE.md files, which means I can probably make the experience even better once I take the time to explore the customization options.

While this is a single-person hobby project, it made me very optimistic about my ability to use LLM in future prototyping situations.
On the negative side, I found the personality sycophantic (“yes, what a great idea!”), and sometimes plain mistaken (“now the case doesn’t have any disconnected parts”).
It tended to invent some facts that fit its observations (“QR codes do not give us orientation position”), or if left to its own devices, it sometimes “solved” problems by mocking out entire procedures (leaving a “TODO” marker) or sprinkling magic constants in the code.

The overall conclusion is that if I was to personify the LLM, it behaves like a university graduate. It has impressive general knowledge, but not enough street smarts to always apply it in a productive way. Some problems it solved beautifully on its own, and some got it to spiral out.

I didn’t let it handle commits in any form. This was a sort of a safety net. It could do what it wants with the files inside the repo, but I would always commit/revert depending on the success of our work.

Future work

Like any hobby project, it’s not done.
I’m waiting on a smaller PSU and some wiring that will allow me to replace the USB cables and hopefully make this look less cluttered.
I’m also thinking about whether the SKÅDIS board is a good enough mount, or should I try doing something more bespoke and elegant.
Software-wise, I’m thinking about making the displayed image depend on something, like the time of day, the weather outside, holidays etc.

I’ll try to post updates whenever I have any really interesting changes.

Introducing ESLint to your codebase smoothly

2025-07-03T00:00:00+03:00

When adding a linter to an existing codbase, my methodology is as follows:

Create lint config files and approve them with the team (people have strong opinions about tab widths), add them to the repo
Run the linter a single time over the entire codebase, autofixing what it can, and adding comments to ignore what it can’t.
Add some pre-commit / pre-merge rule to run the linter and preventing new violations being added
Grepping the ignore comments we added in #2, fixing their violations carefully

While it might seem a bit complex, I like it because

Each commit is easy to review - either small or non-risky (adding comments)
There are no “no one commit, I’m doing a thing” moments that sometimes occur when refactoring
We’re not blocked on “fixing all violations” before introducing linting

Ruff is great for this as it has --add-noqa, which adds “ignore” comments over every violation it finds. I recently had to do the same with ESLint, which doesn’t have that feature.

I had to improvise.

Solution

With some LLMing, I composed a script that iterates over ESLint’s errors, and adds an “ignore on next line” comment before it

// add_comments.js
const fs = require('fs');
const path = require('path');

const errorsFilePath = process.argv[2];
if (!errorsFilePath) {
    console.error('Please provide the path to the eslint errors JSON file');
    process.exit(1);
}

const errors = JSON.parse(fs.readFileSync(errorsFilePath, 'utf-8'));
errors.forEach(file => {
    const filePath = path.resolve(file.filePath);
    let content = fs.readFileSync(filePath, 'utf-8');
    const lines = content.split('\n');

    const rulesMap = new Map();
    // Multiple rules per line
    file.messages.forEach(message => {
        const line = message.line;
        const ruleId = message.ruleId;

        if (!rulesMap.has(line)) {
            rulesMap.set(line, []);
        }
        rulesMap.get(line).push(ruleId);
    });

    // Reverse order to avoid spoiling next lines
    for (let line of Array.from(rulesMap.keys()).sort((a, b) => b - a)) {
        const ruleIds = rulesMap.get(line).join(', '); // Join rule IDs for the comment
        const comment = `// eslint-disable-next-line ${ruleIds}`;
        lines.splice(line - 1, 0, comment);
    }

    content = lines.join('\n');
    fs.writeFileSync(filePath, content, 'utf-8');
});

It’s then used like this:

git checkout :/ && npx eslint --format json > complaints.json ; node /tmp/add_comments.js complaints.json && npx eslint

Notable items about the script:

Crude way to select an input file. I miss argparse
Grouping the errors by line, as one line might have multiple errors
Applying the fixes in reverse, as adding new lines will skew the line numbers of the following lines
Not caring about indentation - this can be handled by e.g. prettier

Solution #2: jsx

Running the same procedure on files containing JSX didn’t work as well, as it produced JS comments in the middle of JSX templates. Consider this:

function best() {
  return (
    <div>
      <ul>
        <li key="item">Item 1</li>
        <li key="item">Item 2</li>
      </ul>
    </div>
  );
}

Naively, it would get processed into

function best() {
  return (
    <div>
      The result of 1 + 1 is: {result}
      <ul>
// eslint-disable-next-line react/jsx-key
        <li key="item">Item 1</li>
        <li key="item">Item 2</li>
      </ul>
    </div>
  );
}

Where the new comment obviously breaks the JSX tempalte. It’d need to be converted into this:

function best() {
  return (
    <div>
      <ul>
        {/* eslint-disable-next-line react/jsx-key */}
        <li key="item">Item 1</li>
        <li key="item">Item 2</li>
      </ul>
    </div>
  );
}

I considered improving the original script to detect whether we’re inside a JSX tag or not, but it was too complicated. Instead, I decided on running ESLint again, picking up its complaints on “you can’t have a comment here” and rewriting the lines referenced.

// jsx_comments.js
const fs = require('fs');
const path = require('path');
const assert = require('assert');

const errorsFilePath = process.argv[2];
if (!errorsFilePath) {
    console.error('Please provide the path to the eslint errors JSON file as an argument.');
    process.exit(1);
}

const errors = JSON.parse(fs.readFileSync(errorsFilePath, 'utf-8'));
errors.forEach(file => {
    const filePath = path.resolve(file.filePath);
    let content = fs.readFileSync(filePath, 'utf-8');
    const lines = content.split('\n');
    const messages = file.messages.filter(m => m.ruleId == "react/jsx-no-comment-textnodes");
    messages.forEach(m => {
      let lineIndex = m.line;
      // Eat some newlines
      while (/^\s*$/.test(lines[lineIndex])) {
        lineIndex++;
      }
      const line = lines[lineIndex];
      assert(/^\s*\/\/ eslint-disable-next-line/.test(line), `line needs to match: ${line}`);
      const newLine = line.replace(/^\/\/ eslint-disable-next-line (.+)$/, '{/* eslint-disable-next-line $1 */}');
      lines[lineIndex] = newLine;
    });

    content = lines.join('\n');
    fs.writeFileSync(filePath, content, 'utf-8');
});

Then

git checkout :/ && npx eslint --format json > jsx_complaints.json ; node /tmp/jsx_comments.js jsx_complaints.json && npx eslint 

Or all at once:

git checkout :/ \
&& npx eslint --format json > complaints.json ; node /tmp/add_comments.js complaints.json \
&& npx eslint --format json > jsx_complaints.json ; node /tmp/jsx_comments.js jsx_complaints.json \
&& npx eslint 

Worked great.

A quick and simple VPN

2025-05-18T00:00:00+03:00

I’m currently on vacation abroad and need access to one of the government-run websites to coordinate a time-sensitive matter. As a very cheap security measure, said government website doesn’t work if accessed from a foreign VPN. A courteous explanation would be that the attack/usage ratio from abroad is much higher. A less favorable one would be that they’re just lazy.

I used my AWS account to access the website really quickly:

Create a tiny server in a physical region that would be accepted by the website. Allow SSH from 0.0.0.0 and external IP, as per the defaults.
Run this to create a local SOCKS proxy over SSH that routes traffic via that server: ssh -i ~/.ssh/aws.pem ubuntu@EXTERNALIP -D 1234
Run Chromium on a guest profile connecting to said SOCKS proxy: chromium --proxy-server="socks5://localhost:1234" --guest. I think one could use this Firefox equivalent (untested): firefox --proxy-server="socks5://localhost:1234" --private-window

Surf away. Don’t forget to terminate the server once you’re done, or else you’ll incur (checks notes…) $3.5 a month.

A Laptop can be a Big Raspberry Pi

2025-03-15T00:00:00+02:00

I used to be one of those people running a rpi home server. I have a long history with running rpi, and I learned some things along the way:

The disasterous effects of undervoltage on attached harddrives (goodbye data integrity, hello fsck on boot)
The difficulty of running an OS from an SDcard (system upgrades take as long to write to disk as they do to download the packages)
The oddities of not having a proper system clock when starting up (tls certs are “not yet valid” because we’re now 5 years into the past)
The permanent contest between GPU and CPU memory allocation when you want to run something GPU intensive

After talking with a friend about my next rpi-related purchase, they commented that I should just get a laptop. I obviously said that this is not the same, and they said “yeah, it’s better”. Their point was that for approximately the same amount of money I invest in my rpi setup, I can get a refurbished T470 off the internet and run whatever I want on it. I agreed to try.
It’s been almost 2 years since that conversation, and I admit I was wrong, and a decent refurbished laptop is a great platform for a home server.
Here are some reasons why I now think laptops are superior to rpis:

Trivially-stacking form factor
Standard power requirements - they might not run on phone chargers, but they’re at very little risk of underdelivering current to their internal components / peripherals
Real clock and battery to surive outages and restarts
x86 cpu means that you’re not running into some edge cases where packages are not compiled properly / not available for ARM
Multiple peripheral ports
(relatively) beefy Bluetooth and Wifi stacks
Embedded KVM if needed, hidden neatly away if not
Proper internal hdd / ssd
Active ventilation. At the very least, you’ll hear when it’s overheating

There are some drawbacks, obviously:

Bigger, although for a single server this is not a biggie, there’s room for it
Requires a PSU brick, although it usually comes with one
Much less cool and DIY-ish, although at some point you’re having enough of custom-writing udev rules and just want things to work
No native support for HDMI-CEC (controlling the laptop via the TV’s remote)

Nowdays, whenever I’m asked to “stick a rpi behind a tv” to create a “smart tv” anywhere, be it at friends’ homes or companies’ expos, I’m asking for an old (but still decent) laptop and using that instead of a tiny ARM munchkin.

Streaming SQL results from SQLALchemy via a FastAPI endpoint

2025-03-11T00:00:00+02:00

I was asked to create an endpoint that gets an SQL query and replies with a JSON list of the results. The prototype was ready in 10 minutes:

@app.post("/sql")
async def sql(
    query: str = Form(),
    user: OpenID = Depends(get_logged_user),
):
    async with session_maker() as session:
        async with session.begin():
            try:
                result = await session.execute(text(query))
            except Exception as e:
                raise HTTPException(status_code=400, detail=str(e))

            keys = result.keys()
            rows = result.fetchall()
            return [
                dict(zip(keys, row))
                for row in rows
            ]

Which was nice, and then promptly crashed the process when we sent a query with a ginormous result set and the Python process tried to fetch all of the rows into memory. One can argue about the validity of large reult sets and mention paging and whatnot, but these were “research” related one-time queries, and therefor merited large result sets and resisted paging etc. I wanted to move into streaming the results (“server side cursors” is the applicable SQLism, I think), but ran into problems where the transaction ended before the streaming response started.

This did not work:

@app.post("/sql")
async def sql(
    query: str = Form(),
    user: OpenID = Depends(get_logged_user),
):
    async with session_maker() as session:
        async with session.begin():
            try:
                result = await session.stream(text(query))
            except Exception as e:
                raise HTTPException(status_code=400, detail=str(e))

            keys = result.keys()
            async def streamy():
                first = True
                yield '['
                for row in result: 
                    if first:
                        first=False
                    else:
                        yield ','
                    yield json.dumps(dict(zip(keys, row)))
                yield ']'
            return StreamingResponse(streamy(), media_type="application/json")

Since the actual invocation of streamy happend outside the with block for the session, SQLAlchemy would complain.

The next step was moving the session into the streamed function:

async def streamy(query):
    async with session_maker() as session:
        async with session.begin():
            result = await session.stream(text(query))
            keys = result.keys()
            first = True
            yield '['
            for row in result: 
                if first:
                    yield ','
                else:
                    first=True
                yield json.dumps(dict(zip(keys, row)))
            yield ']'

@app.post("/sql")
async def sql(
    query: str = Form(),
    user: OpenID = Depends(get_logged_user),
):
    return StreamingResponse(streamy(query), media_type="application/json")

The above worked, but only when the SQL query was successful. Once streaming was started, the response is going to be a 200 response even if an error was thrown. There was no way to report an error back to the user, aside from putting it inside the json response which will be surprising for unsuspecting clients.

The version I was happy with gave up on the with block, and instead explicitly opened and closed the SQLAlchemy session. The closing only happened inside the streaming function, which ensured the session was available for the duration of the stream. Executing the query before returning a StreamingResponse allows me to still return a 400 if the query runs into trouble

@app.post("/sql")
async def sql(
    query: str = Form(),
    user: OpenID = Depends(get_logged_user),
):
    session = session_maker()
    await session.begin()
    try:
        result = await session.stream(text(query))
    except Exception as e:
        raise HTTPException(status_code=400, detail=str(e))

    keys = result.keys()
    async def streamy():
        try:
            yield '['
            first = True
            async for row in result:
            for row in result: 
                if first:
                    first=False
                else:
                    yield ','
                yield json.dumps(dict(zip(keys, row)))
            yield ']'
        finally:
            await session.close()

    return StreamingResponse(streamy(), media_type="application/json")

Happy days

Google Storage using cURL

2024-10-11T00:00:00+03:00

I got around to troubleshooting a Python process running in Docker that had some permission problems accessing Google Storage Reproing inside Python with google-cloud-storage was a bit bothersome, and installing the gcloud cli on an Ubuntu-based Docker image takes too much effort on an ephemeral container (if I change stuff, I need to create a new container and I’ll have to reinstall it all over again). I was looking for something that is both easy to run, and doesn’t require me to modify the base container image (the build pipeline was annoying to mess with).

cURL to the rescue!

Getting an access token

This fetches a token from GKE’s metadata endpoint.

TOKEN="$(curl -H "Metadata-Flavor: Google" 'http://metadata/computeMetadata/v1/instance/service-accounts/default/token' -s | jq .access_token -r)"

Listing files in a bucket

curl -X GET -H "Authorization: Bearer $TOKEN" "https://storage.googleapis.com/storage/v1/b/$BUCKET_NAME/o/"

Getting a file

Note that FILENAME needs to be url-encoded, even the slashes that are used for “directories”.
Without ?alt=media, you’ll get metadata for the file

curl -X GET -H "Authorization: Bearer $TOKEN" "https://storage.googleapis.com/storage/v1/b/$BUCKET_NAME/o/$FILENAME?alt=media"

Putting a file

Again, FILENAME needs to be properly encoded

curl -X POST --data-binary @ACTUAL_FILE_PATH \
    -H "Authorization: Bearer $TOKEN" \
    "https://storage.googleapis.com/upload/storage/v1/b/$BUCKET_NAME/o?uploadType=media&name=$FILENAME"