@movq Wow, I use Firefox and didn't realize this existed! Thanks for pointing it out. I noticed at least one bug cited a webcompat.com report; I wonder if someone at Mozilla monitors those. https://webcompat.com/issues?page=1&per_page=50&state=open&stage=all&sort=created&direction=desc

@javivf Welcome to twtxt!

@lyse Thanks for taking a look, and for pointing out the mixture of tabs and spaces.

I think I'll leave reachability.c alone, since my intention there was to use an indent level of one tab, and the spaces are just there to line up a few extra things. I fixed reachability_with_stack.cc though.

@prologic Those aren't actually serving anything public-facing. I've thought about it, but for now I'm sticking with VPSs, partly because I don't relish the risk of weeks of downtime if something goes wrong while I'm travelling.

@prologic Here's mine.

@lyse I don't remember exactly. They might have been growing all winter. The trick is to have a badly insulated extension to the house.

@lyse I am a big fan of "obvious" math facts that turn out to be wrong. If you want to understand how reusing space actually works, you are mostly stuck reading complexity theory papers right now. Ian wrote a good survey: https://iuuk.mff.cuni.cz/~iwmertz/papers/m23.reusing_space.pdf . It's written for complexity theorists, but some of will make sense to programmers comfortable with math. Alternatively, I wrote an essay a few years ago explaining one technique, with (math-loving) programmers as the intended audience: https://www.falsifian.org/blog/2021/06/04/catalytic/ .

@lyse Still melting!

@sorenpeter Sorry, I realized that shortly after posting. Here's another attempt to post the images:

@prologic Have you tried Google's robots.txt report? https://support.google.com/webmasters/answer/6062598?hl=en . I would expect Google to be pretty good about this sort of thing. If you have the energy to dig into it and, for example, post on support.google.com, I'd be curious to hear what you find out.

@eapl.me I like this idea. Another option would be to show a limited number of posts, with an option to see the omitted ones by user. Either way, I wonder how well that works with threading.

@lyse Thanks for sharing. I really enjoyed it. The beginning part about the history of life on Earth was fun to watch having just read Dawkin's old book The Selfish Geene, and now I want to read more about archaea. The end of the talk about what might be going on on Mars made me a bit hopeful someone will find some good evidence.

@jost Hello jost!

@movq Looks fun. Also kind of looks like APL and Forth had a baby on Jupyter.

@andros Hello!

@kh1b Welcome to twtxt!

@wbknl Welcome to the twtxt-iverse!

@lyse Beautiful pictures, and beautiful HTML for a photo album!

@prologic I'm grateful for this accident. I find browsing twtxt.net useful even though I don't have an account there. I do it when I can't use Jenny because I only have my phone, or if I want to see messages I might have missed. I know it's not guaranteed to catch everything, but it's pretty good, even if it's not intentional.

@Codebuzz I use Jenny to add to a local copy of my twtxt.txt file, and then manually push it to my web servers. I prefer timestamps to end with "Z" rather than "+00:00" so I modified Jenny to use that format. I mostly follow conversations using Jenny, but sometimes I check twtxt.net, which could catch twts I missed.

@bender I try to avoid editing. I guess I would write 5/4, 6/4, etc, and hopefully my audience would be sympathetic to my failing.

Anyway, I don't think my eccentric decision to number my twts in the style of other social media platforms is the only context where someone might write 1/4 not meaning a quarter. E.g. January 4, to Americans.

I'm happy to keep overthinking this for as long as you are :-P

@bender @prologic I'm not exactly asking yarnd to change. If you are okay with the way it displayed my twts, then by all means, leave it as is. I hope you won't mind if I continue to write things like 1/4 to mean "first out of four".

What has text/markdown got to do with this? I don't think Markdown says anything about replacing 1/4 with ¼, or other similar transformations. It's not needed, because ¼ is already a unicode character that can simply be directly inserted into the text file.

What's wrong with my original suggestion of doing the transformation before the text hits the twtxt.txt file? @prologic, I think it would achieve what you are trying to achieve with this content-type thing: if someone writes 1/4 on a yarnd instance or any other client that wants to do this, it would get transformed, and other clients simply wouldn't do the transformation. Every client that supports displaying unicode characters, including Jenny, would then display ¼ as ¼.

Alternatively, if you prefer yarnd to pretty-print all twts nicely, even ones from simpler clients, that's fine too and you don't need to change anything. My 1/4 -> ¼ thing is nothing more than a minor irritation which probably isn't worth overthinking.

@prologic I'm not a yarnd user, so it doesn't matter a whole lot to me, but FWIW I'm not especially keen on changing how I format my twts to work around yarnd's quirks.

I wonder if this kind of postprocessing would fit better between composing (via yarnd's UI) and publishing. So, if a yarnd user types 1/4, it could get changed to ¼ in the twtxt.txt file for everyone to see, not just people reading through yarnd. But when I type 1/4, meaning first out of four, as a non-yarnd user, the meaning wouldn't get corrupted. I can always type ¼ directly if that's what I really intend.

(This twt might be easier to understand if you read it without any transformations :-P)

Anyway, again, I'm not a yarnd user, so do what you will, just know you might not be seeing exactly what I meant.

@prologic I wrote 1/4 (one slash four) by which I meant "the first out of four". twtxt.net is showing it as ¼, a single character that IMO doesn't have that same meaning (it means 0.25). Similarly, 3/4 got replaced with ¾ in another twt. It's not a big deal. It just looks a little wrong, especially beside the 2/4 and 4/4 in my other two twts.

@prologic One could argue twtxt.net's display formatting is a little over-eager here.

@prologic I think printf is a more portable option than echo -e for interpreting \t as tab. E.g. printf '%s\t%s\t%s' "$url" "$time" "$text". In general I always prefer printf over echo for anything non-trivial in unix shell scripts. See last paragraph of https://en.wikipedia.org/wiki/Echo_(command)#History

@aelaraji You could just remove the {getuser()} part because you added ~.

@bender It's the experience of an ordinary person in a strange place where memories are disappearing with the help of the Memory Police. The setting feels contemporary (to the book's 1994 publication date) rather than futuristic, except for some unexplained stuff about memories.

@prologic

See https://dev.twtxt.net

Yes, that is exactly what I meant. I like that collection and "twtxt v2" feels like a departure.

Maybe there's an advantage to grouping it into one spec, but IMO that shouldn't be done at the same time as introducing new untested ideas.

See https://yarn.social (especially this section: https://yarn.social/#self-host) -- It really doesn't get much simpler than this 🤣

Again, I like this existing simplicity. (I would even argue you don't need the metadata.)

That page says "For the best experience your client should also support some of the Twtxt Extensions..." but it is clear you don't need to. I would like it to stay that way, and publishing a big long spec and calling it "twtxt v2" feels like a departure from that. (I think the content of the document is valuable; I'm just carping about how it's being presented.)

@prologic Done. Also, I went ahead and made two changes: changed hexadecimal to base64 for hashes (wasn't sure if anyone objected), and changed "MUST follow the chain" to "SHOULD follow the chain.

@prologic Thanks for pointing out it lasts four hours. That's a big window! I wonder when most people will be on. I might aim for halfway through unless I hear otherwise. (12:00Z is a bit early for me.)

@movq Yes, the tools are surprisingly fast. Still, magrep takes about 20 seconds to search through my archive of 140K emails, so to speed things up I would probably combine it with an indexer like mu, mairix or notmuch.

@bender Ha! Maybe I should get on the Markdown train. You're taking away my excuses.

@bender

Sorry, you're right, I should have used numbers!

I'm don't understand what "preserve the original hash" could mean other than "make sure there's still a twt in the feed with that hash". Maybe the text could be clarified somehow.

I'm also not sure what you mean by markdown already being part of it. Of course people can already use Markdown, just like presumably nothing stopped people from using (twt subjects) before they were formally described. But it's not universal; e.g. as a jenny user I just see the plain text.

@prologic Do you feel the same about published vs. privately stored data?

For me there's a distinction. I feel very strongly that I should be able to retain whatever private information I like. On the other hand, I do have some sympathy for requests not to publish or propagate (though I personally feel it's still morally acceptable to ignore such requests).

@lyse I'd suggest making the whole content-type thing a SHOULD, to accommodate people just using some hosting service they don't have much control over. (The same situation could make detecting followers hard, but IMO "please email me if you follow me" is still legit twtxt, even if inconvenient.)

@prologic Thanks for writing that up!

I hope it can remain a living document (or sequence of draft revisions) for a good long time while we figure out how this stuff works in practice.

I am not sure how I feel about all this being done at once, vs. letting conventions arise.

For example, even today I could reply to twt abc1234 with "(#abc1234) Edit: ..." and I think all you humans would understand it as an edit to (#abc1234). Maybe eventually it would become a common enough convention that clients would start to support it explicitly.

Similarly we could just start using 11-digit hashes. We should iron out whether it's sha256 or whatever but there's no need get all the other stuff right at the same time.

I have similar thoughts about how some users could try out location-based replies in a backward-compatible way (append the replyto: stuff after the legacy (#hash) style).

However I recognize that I'm not the one implementing this stuff, and it's less work to just have everything determined up front.

Misc comments (I haven't read the whole thing):

• Did you mean to make hashes hexadecimal? You lose 11 bits that way compared to base32. I'd suggest gaining 11 bits with base64 instead.

• "Clients MUST preserve the original hash" --- do you mean they MUST preserve the original twt?

• Thanks for phrasing the bit about deletions so neutrally.

• I don't like the MUST in "Clients MUST follow the chain of reply-to references...". If someone writes a client as a 40-line shell script that requires the user to piece together the threading themselves, IMO we shouldn't declare the client non-conforming just because they didn't get to all the bells and whistles.

• Similarly I don't like the MUST for user agents. For one thing, you might want to fetch a feed without revealing your identty. Also, it raises the bar for a minimal implementation (I'm again thinking again of the 40-line shell script).

• For "who follows" lists: why must the long, random tokens be only valid for a limited time? Do you have a scenario in mind where they could leak?

• Why can't feeds be served over HTTP/1.0? Again, thinking about simple software. I recently tried implementing HTTP/1.1 and it wasn't too bad, but 1.0 would have been slightly simpler.

• Why get into the nitty-gritty about caching headers? This seems like generic advice for HTTP servers and clients.

• I'm a little sad about other protocols being not recommended.

• I don't know how I feel about including markdown. I don't mind too much that yarn users emit twts full of markdown, but I'm more of a plain text kind of person. Also it adds to the length. I wonder if putting a separate document would make more sense; that would also help with the length.

@prologic I have no specifics, only hopes. (I have seen some articles explaining the GDPR doesn't apply to a "purely personal or household activity" but I don't really know what that means.)

I don't know if it's worth giving much thought to the issue unless either you expect to get big enough for the GDPR to matter a lot (I imagine making money is a prerequisite) or someone specifically brings it up. Unless you enjoy thinking through this sort of thing, of course.

@david Thanks, that's good feedback to have. I wonder to what extent this already exists in registry servers and yarn pods. I haven't really tried digging into the past in either one.

How interested would you be in changes in metadata and other comments in the feeds? I'm thinking of just permanently saving every version of each twtxt file that gets pulled, not just the twts. It wouldn't be hard to do (though presenting the information in a sensible way is another matter). Compression should make storage a non-issue unless someone does something weird with their feed like shuffle the comments around every time I fetch it.

@movq I don't think it has to be like that. Just make sure the new version of the twt is always appended to your current feed, and have some convention for indicating it's an edit and which twt it supersedes. Keep the original twt as-is (or delete it if you don't want new followers to see it); doesn't matter if it's archived because you aren't changing that copy.

@prologic Do you have a link to some past discussion?

Would the GDPR would apply to a one-person client like jenny? I seriously hope not. If someone asks me to delete an email they sent me, I don't think I have to honour that request, no matter how European they are.

I am really bothered by the idea that someone could force me to delete my private, personal record of my interactions with them. Would I have to delete my journal entries about them too if they asked?

Maybe a public-facing client like yarnd needs to consider this, but that also bothers me. I was actually thinking about making an Internet Archive style twtxt archiver, letting you explore past twts, including long-dead feeds, see edit histories, deleted twts, etc.

@david Well, I wouldn't recommend using my code for your main jenny use anyway. If you want to try it out, set XDG_CONFIG_HOME and XDG_CACHE_HOME to some sandbox directories and only run my code there. If @movq is interested in any of this getting upstreamed, I'd be happy to try rebasing the changes, but otherwise it's a proof of concept and fun exercise.

I forgot to git add a new test file. Added to the patch now at https://www.falsifian.org/a/oDtr/patch0.txt

@david Hello!

BTW this code doesn't incorporate existing twts into jenny's database. It's best used starting from scratch. I've been testing it using a custom XDG_CACHE_HOME and XDG_CONFIG_HOME to avoid messing with my "real" jenny data.

@prologic Wikipedia claims sha1 is vulnerable to a "chosen-prefix attack", which I gather means I can write any two twts I like, and then cause them to have the exact same sha1 hash by appending something. I guess a twt ending in random junk might look suspcious, but perhaps the junk could be worked into an image URL like . If that's not possible now maybe it will be later.

git only uses sha1 because they're stuck with it: migrating is very hard. There was an effort to move git to sha256 but I don't know its status. I think there is progress being made with Game Of Trees, a git clone that uses the same on-disk format.

I can't imagine any benefit to using sha1, except that maybe some very old software might support sha1 but not sha256.

@movq Agreed that hashes have a benefit. I came up with a similar example where when I twted about an 11-character hash collision. Perhaps hashes could be made optional somehow. Like, you could use the "replyto" idea and then additionally put a hash somewhere if you want to lock in which version of the twt you are replying to.

@quark Oh, sure, it would be nice if edits didn't break threads. I was just pondering the circumstances under which I get annoyed about data being irrecoverably deleted or otherwise lost.

@quark I don't really mind if the twt gets edited before I even fetch it. I think it's the idea of my computer discarding old versions it's fetched, especially if it's shown them to me, that bugs me.

But I do like @movq's suggestion on this thread that feeds could contain both the original and the edited twt. I guess it would be up to the author.

@quark None. I like being able to see edit history for the same reason.