Week 39-41, 2025 - Restore
I'm trying to get back to a regular cadence of updates, and will try fortnightly next. (Are you also the person who actively avoids "biweekly" and uses "twice a week" and "fortnightly" instead, because, however ridiculously obvious it would be from the context, you still want to be extra-super sure nobody misunderstands you? Welcome to the club.)
Recently, I spotted an interesting trend when working with chat-based LLMs. The answer delay (especially on Claude - ChatGPT seems a bit faster) is a huge distraction trap! I send a well-crafted prompt, multiple lines sometimes, and then during the wait, even if it's just a few seconds, my mind starts to wander. I open Reddit, check Slack for new messages, refresh a news site, stock prices, emails, anything that feeds dopamine to my stimulation-hungry brain. Then an hour passes, and I don't even remember what I wanted to do with the LLM.
Here's what I'm trying to do now instead:
I stay waiting.
I use these boring seconds to breathe consciously, think about my next steps in the project, or step back and decide if I'm really going in the direction I wanted to, or am just lost in a rabbit hole of curiosity. If I were to work in an office, I could swordfight while balancing on a roller chair, too – even that's a better alternative to doomscrolling.
📋 What I learned recently
This was a rollercoaster. Last week, I almost lost most of my digital photos, family videos, scanned documents from the last 15 years, backups, archives, and a small music and movie collection. Here's what happened.
I have a nice small-form-factor PC now, so two weeks ago I moved the previously USB-connected 4TB HDD that stores basically all my personal data internally. However, I noticed that write speeds are extremely low, like 6MB/sec, compared to 100+ for reading. I started to debug with Claude AI, and changing the connection setting from SATA to SCSI solved the problem nicely. This was working fine for a while, but then I tried playing with the spindown time settings for the HDD to achieve quiet operation, and immediately, write speeds fell to below 10MB/s again.
Continuing the Claude discussion from where we fixed the problem before, I explained my recent changes and asked for help diagnosing the issue. Here's how it went:
See if you can spot the problem.
Well, I couldn't, and the combination of earlier successes and being distracted listening in on a meeting while working on my homelab (what was I thinking?!) made me drop my guard and blindly copy-paste the command that overwrote the beginning of the HDD device with zeros.
I pressed Ctrl+C after a few seconds, but I figured the damage was done. Claude agreed:
What followed was a frantic scramble for solutions to save the day, but when I realized far more than "just" the partition table was overwritten, so even its in-memory copy wouldn't help, I gave up and accepted the reality of an unplanned backup restoring session.
Which... went great! Restic is amazing, it effortlessly restored all ~2TB of data (I didn't do a detailed damage assessment, just restored everything) in a day or so. Even user/group ownership info and other metadata were preserved perfectly. I was prepared for a higher Backblaze bill this month, and became comfortable accepting it as a reminder to be more vigilant, but to my surprise, nothing extra was charged: up to three times of average storage size worth egress is included without charge in B2 storage costs. Awesome!
Lessons:
- Don't multitask.
- Don't type anything into a prompt that you don't understand.
- Have validated, up-to-date backups at hand.
🧑💻 Something cool: Sipeed NanoKVM-PCIe
Still homelabbing: Since installing it, about once or twice a week, my home server disconnected from the network without any obvious signs. Some googling / Claude investigation led to a known problem: the Intel e1000 chipset under Linux can randomly get into a hung state, especially during high data throughput. A simple ifup/ifdown reconnects the device until the next incident, but of course, it requires sitting in front of the console. To avoid having to do that, I wanted an easy way to diagnose and fix problems remotely, and after some research, I ordered the PCIe version of Sipeed's NanoKVM.
The device, which is almost identical to their original Cube version, connects to motherboard pins within my server to allow remote power up/down, an HDMI and USB port to allow screen and keyboard/mouse access, and ethernet to my ISP's router, so I can access it even if my own router is down. It even has some storage for iso images, which I can mount remotely and boot the computer from.
Here's my PC running a NanoKVM-mounted SystemRescue disc, in a Chrome window on my laptop, sitting in an office connecting to my home network via my router's VPN:
That’s it for today, think about your backup strategy this weekend,
Péter