Pseudpocalypse

(dynomight.net)

151 points | by surprisetalk 11 days ago

33 comments

  • Rendello 9 days ago
    There's been some interesting threads about stylometry over the years [1]. The top link was quite decent at unmasking HN alt accounts with basic ngram analysis whipped up in one day [2].

    1. https://hn.algolia.com/?q=stylometry

    2. https://news.ycombinator.com/item?id=33756141

    • tarpitt 9 days ago
      I remember this! it got me pretty good. I have a bad habit of generating alts because I forget the password.

      It makes me wonder if we could use non-instruct LLMs to slightly alter the wording of text while keeping the meaning the same meaning. Perhaps by using perplexity or some other metric. I don't know, maybe you compare the distance of the "meaning" vectors.

      You might also want to have some "style" vector associated with each pseudonym. For example, I might want it to produce british english under a certain pseudonym, and simulate an ESL speaker under another.

      Essentially, you would want some way of re-styling text. The basic way to do this would be to run the same sylometry tools the hunter uses and manually make synonym word/phrase subsitutions to lower your similarity.

      It's a cat and mouse game, but I think the mouse eventually wins. Consider a program that translates your english writing programmaticially into a low-entropy symbolic form and then translates them back to english in a procedural manner. Basicially you design an intermediary language that cannot contain style. It would be boring to read but it would remove all the style.

      • Alive-in-2025 9 days ago
        Yep, I was thinking the exact same thing. Write something, get your llm to rewrite it to "standard Steve Yegge form" to hide your identity. If we all do the same we are probably safe.
    • thinking_cactus 9 days ago
      If that's as effective as comments suggest, then it seems indeed unmasking pseudonyms has only been a matter of effort for a while.

      I still think people underestimate the power of even minor inconvenience. While you can't just click a button to reveal all pseudonyms of someone (e.g. you need to download some obscure tool or even perform statistical analysis yourself), I think this provides significant (and surprising?) protection for pseudonymous individuals, I'd say even (although to a reduced extent) for sophisticated threats like state-level actors. I hope LLMs continue to be unable to do so for the near future (I just tested and LLMs can't do it with a simple prompt).

      Which is why I think privacy safeguards still work quite well even while being technically mostly bypassable.

      So probably tools like this should be kept private if possible.

    • saghm 9 days ago
      Looks like the site was taken down. I was curious who wrote the most like me, given that I don't have any alt accounts, but I guess weighing privacy over my curiosity is a good thing in the big picture
  • Cynddl 9 days ago
    Entropy is unfortunately a very bad metric to estimate if these identification techniques will scale. Plugging my work here as example: https://www.nature.com/articles/s41467-024-55296-6
  • YeGoblynQueenne 9 days ago
    "Pseudpocalypse"... why not "pseudocalypse" or "pseudoapocalypse"?

    As an aside, it's always surprising to see how English speakers split Greek words like "Apocalypse". That is to say, they always split them in the middle of Greek syllables, or just drop letters like "pseud[o][a]pocalypse" and often in a way that ends up sounding clunky and weird even in English.

    Can't think of other examples now. Brain going to sleepzzzz....

    Edit: oh wow there's actually a word for that:

    https://en.wikipedia.org/wiki/Libfix

    OK, now I go to sleep.

    • dooglius 9 days ago
      "pseud" is it's own atom here, see definition 2 in https://www.merriam-webster.com/dictionary/pseud

      It's not trying to say "something like an apocalypse" it's saying "an apocalypse of pseuds"

      • viccis 9 days ago
        Yeah, pseudpocalypse would be Substack going out of business or something similar.
    • Exoristos 9 days ago
      Let's see, expanded it's "the (or an) apocalypse of pseudonyms." In Greek, this would roughly be "ἀποκάλυψις ὧν ψευδωνῠ́μων." It's not impossible for a genitive like this to become a word, possibly "ἀποκάλυψευδώνῠμοι" (think of "φιλέλλην") -- although it's a little fanciful. There may even be precedent for a construct that puts "ψευδώνῠμοι" first. But ultimately I agree that the word the article offers seems tortured.
      • YeGoblynQueenne 8 days ago
        Πολυτονικό! Haha! But yeah, αποκαλυψευδώνυμοι/α sounds better, but it would work even less well in English.
    • Schlagbohrer 8 days ago
      English speakers even mix latin and greek roots!

      From a t-shirt I saw once: "I am against polyamory! It should either be multiamory or polyphilia!"

    • Avicebron 9 days ago
      It's a speech and attention thing. With the American accent you weigh the pseud part higher than the -onym because it's the downward part of the word. pseudocalypse highlights the -ocalypse on the upswing which detracts from the pseudo- so to maximize traction use both recognition highlights pseud- and -pocalypse.
  • totetsu 9 days ago
    To achieve this level of total surveillance it would trillions and trillions of dollars to be invested in a massive roll out of data centers. I just dont see it happening.
    • dozerly 9 days ago
      Right now… it would have been science fiction 10 years ago and not a trillion dollar problem.
      • totetsu 8 days ago
        10-15 Years ago it was a was a really fun limited Comic series by Brian K. Vaughan, Marcos Martin and Muntsa Vicente, called The Private Eye. About a world where the Cloud had Burst, and leaked 100% of everyones information, leading to an internet less society where eveyone goes around wearing disguises. https://www.digitalamerica.org/cloud-burst-review-brian-k-va...
    • King-Aaron 9 days ago
      > I just dont see it happening.

      It is though.

      • Schlagbohrer 8 days ago
        whoosh

        Or to quote Drax the Destroyer: "A joke would never fly over my head! My reflexes are too good. I would catch it. :-/ "

      • bee_rider 9 days ago
        I’m 95% sure that was sarcasm.
  • MassPikeMike 9 days ago
    Perhaps remarkably, the US intelligence community is funding work on this problem in the open, as part of IARPA HIATUS [1].

    The program's initial phase is winding down now, so some of the performers' papers ought to be hitting the ArXiV before too many more months.

    [1] https://www.iarpa.gov/research-programs/hiatus

    • Schlagbohrer 8 days ago
      [dead]
      • MassPikeMike 8 days ago
        If you scroll down to "Prime Performers" on the page I linked, it lists a Massachusetts company and a USC lab that turns out to have a branch office in Massachusetts.
  • jackbravo 9 days ago
    Related articles:

    - Claude knows who you are: https://www.lesswrong.com/posts/Jkb4CBB7rf4XYP5eb/claude-kno...

    - ~ Opus 4.7 is the first model to correctly guess who I am based on unpublished articles: https://x.com/KelseyTuoc/status/2044962428547695007

  • two-sandwich 9 days ago
    I wondered something almost the opposite the other day. With consistent interaction with chat bots and AI output, their linguistic ticks are surely bleeding into every day speech ("smoking gun" and other turns of phrase). What if our linguistic output starts to conform?

    > Do they, incorrectly, position their adverbial clauses? Underrated line.

    • scubbo 9 days ago
      I'd seen the articles making claims about LLM-idiom being unconsciously imitated by humans, and reassured myself that _I_ couldn't possible be so impressionable.

      Then during a presentation I was making today, a coworker wrote in chat "If <scubbo> says 'You're absolutely right' one more time, I'm checking his house to see if we're actually talking to an agent".

      They got me.

    • iammjm 7 days ago
      Our linguistic output definitely IS conforming. It would take our individual sustained efforts not to, and even then it works be hard for is to not to… language is a meme and AI is very good at infecting us with all kinds of memes
    • doitLP 9 days ago
    • rel_ic 9 days ago
      That whole paragraph is clever :)
  • nacozarina 9 days ago
    I've gotten several dozen accounts banned on reddit over the years. I don't remember the names of them all. It would be hilarious to me if someone mined the archives and stitched them all together.
  • kibwen 9 days ago
    As long as you're fine with losing your distinctive voice (which should be taken as table stakes for people who value anonymity to the extent of worrying about this), it's perfectly reasonable to use tools to stymie stylometry. I'm not even suggesting using an LLM (which may or may not be sufficient, it would be difficult to verify either way); it would suffice to have a tool that rewrites your prose (or analyzes it and flags it for manual correction, etc.) if it isn't written in, say, a form where every sentence is primitive subject-verb-object, limited to the 1,000 most common English words, with no contractions, idioms, or exotic punctuation. Yes, this doesn't completely eliminate all possibly identifying bits of entropy, but it would more than suffice for hiding in a crowd ("hiding" in the sense of obviously standing out as someone trying not to be noticed, and as long as you're also careful about your opsec in other ways, like time of posting, etc).
  • largbae 9 days ago
    So who is Satoshi Nakamoto?
  • dfee 9 days ago
    then you'd imagine the inverse is possible: fabricate text that matches a fingerprint - poisoning someone's (psuedo-anonymous) reputation.
  • GMoromisato 9 days ago
    I know I'm a weirdo, but I've been posting under my own name since the 90s.

    I guess I always figured that nobody gives a shit who I am.

    • thomassmith65 9 days ago
      There's plenty of other people on the internet with identities worth stealing. You'll probably be fine. Still, it's not a lottery I want to play.
      • Schlagbohrer 8 days ago
        What happened to the first 64 thomassmiths?
  • negura 8 days ago
    those deanonymization vectors (stylometry and input device biometrics) have been known for a while in the privacy community. re: stylometry, AI rewriting will be useful once it becomes fast & cheap. but research shows that trying to emulate another writing style is already effective. for input device biometrics, there is the kloak algorithm. and which ships with qubesos https://github.com/QubesOS/qubes-gui-daemon/pull/149
  • tarpitt 9 days ago
    I wonder if this could be used to unmask satoshi. I remember a piece about applying stylometry to satoshi's writing, but they just compared him to the usual list of suspects (finney, back, etc.)
    • akk0 8 days ago
      If you help fund the research team, I'll contribute the $5 wrench, and we can split the the profits
  • lnrd 9 days ago
    One would think that it could be possible to make a tool that takes some text and "anonymizes" it by making it a little more standard and boring (uniforming punctuation and sentence structure, changing words with some synonyms, etc). Maybe wouldn't make it particularly compelling, but would be valuable for political dissidents and other people with a high threat model.

    Does anyone have some tools to share?

  • Schlagbohrer 8 days ago
    I wonder how this would work if, for example, someone who never uses AI for writing decides to sometimes use AI for a sentence or string of sentences here and there. Or vice versa, an AI addict makes themselves hand write half the sentences sometimes.

    Basically, putting a pebble in one's shoe to fool gait recognition, what's the equivalent thing for defeating stylometry?

  • fragmede 9 days ago
    That gives a new reason that people should read LLM generated articles. For privacy purposes, if writing is run through Claude to add LLM-isms, then it's harder to deanonymize the writer. If Edward Snowden were posting files online, he'd want to post his writing that way too.
  • pmdulaney 11 days ago
    So anonymity of written speech is toast. We should, however, strive to preserve other forms of anonymity. For example, donations given to political causes should be kept confidential. Let protesters wear masks up to the point where they break the law.
    • RetroTechie 10 days ago
      > So anonymity of written speech is toast.

      No just use a text-mixer: in goes your text, set parameters to have output match <scapegoat> (or just pick "standard_Neanderthal_3"), out comes text conveying the message you wrote, in the style of your choosing.

      Of course that would also strip the attributes that made it your creation.

    • inigyou 9 days ago
      Hmm, the Kill Puppies Movement received $500,000,000 from an anonymous donor.
      • pmdulaney 8 days ago
        And the ACLU famously defended the right of Nazis to parade through Skokie, Illinois.
        • defrost 8 days ago
          It's not inconsistent to

          * support noxious Nazi's speaking out loud and proud in the light of public day so that they may be judged and weighed on their "speech"

          * despise cockroaches crawling in the dark influencing publicly elected representatives with non public "hidden speech" .. a euphemism for bags of life changing amounts of case.

          One is speech, out and proud, the other .. just a bribe, hidden from light.

  • 0zer0 8 days ago
    There's a new 3Blue1Brown video hinting in the same direction.

    https://www.youtube.com/watch?v=GlYgs6v2YfU

  • whodatbo1 9 days ago
    Sooo, this implies such deep profiling hasn't been in use for a decade for target advertisements. 500M is not that large a number with the amount of traces we leave behind online :)
    • DroneBetter 9 days ago
      maybe by the platforms based on the non-publicly-disclosed behaviour (including usage patterns, like rates of opening specific menus), but not by other unprivileged users seeking to publicly out pseudomymous authors.
  • moralestapia 9 days ago
    What a phenomenal article, wow.

    This is one of the best things I've read on this site.

    I have a side project/experiment that's tangential to this (wafertown.com), so my interest is 2x the usual.

  • cyanydeez 9 days ago
    so really, all these people using LLMs to comment aren't being lazy! No, they're using cryptographically linguistic security to ensure untraceability!
  • erelong 11 days ago
    yes but no, can't we just ask AI to sufficiently shuffle our words or for algos to do so?

    "boom", pseudoanonymity (spell?) restored?

    • tacostakohashi 9 days ago
      for future words, sure. but lots of people have an extensive volume of word they already published with the hope / expectation of remaining anonymous.
  • Recursing 9 days ago
    See the classic Gwern post: https://gwern.net/death-note-anonymity

    Which quotes Tao on using deliberate disinformation to preserve anonymity.

    > …one additional way to gain more anonymity is through deliberate disinformation. For instance, suppose that one reveals 100 independent bits of information about oneself. Ordinarily, this would cost 100 bits of anonymity (assuming that each bit was a priori equally likely to be true or false), by cutting the number of possibilities down by a factor of 2100; but if 5 of these 100 bits (chosen randomly and not revealed in advance) are deliberately falsified, then the number of possibilities increases again by a factor of (100 choose 5) ~ 226, recovering about 26 bits of anonymity.

    Intentionally adding writing "tics", scheduling posts to appear between 2am and 6am in your timezone, or pretending to have a different gender/location/age should help a lot in staying pseudonymous for a while longer.

    • strken 9 days ago
      I was thinking this. Why not just lie? Come up with personas, stack them on other personas, and eventually you're pretending to be a 23 year old Asian man from California who is pretending to be a 50 year old New Zealand carpenter and half the bits are muddled.
      • fragmede 9 days ago
        What are you writing about? The real person is going to write about what they know, so when the topic of housing comes up, and they pick New York City to discuss zoning restrictions, sure it's possible they're in New Zealand or California, but it makes more sense to assume the user behind the keyboard is in NYC. The question is, for what purposes is the user trying to be unmasked? In the world we live in, the entirely boring purpose is to sell them ads.
        • strken 8 days ago
          > The question is, for what purposes is the user trying to be unmasked?

          I disagree that ads are the only reason to dox someone. I assume that few pieces of writing would be controversial enough to warrant a fake identity, but those few would be important.

          If I wanted to pseudonymously accuse a mayor of corruption, I think it would be safer to generate true stats and interview real people, but do so in an area I don't live, with some anecdotes where the facts don't quite match my life, using language that has been edited by a friend with a different writing style.

        • Schlagbohrer 8 days ago
          This is why I follow subreddits for cities I have never been to.
    • pixl97 9 days ago
      Possibly, but it can be very difficult to conceal age or location reliability, especially as your corpus of text grows. And of course that is with just the text, when you start adding in other bits of digital information it gets even more difficult.
  • inigyou 9 days ago
    This always seems theoretical. Has it happened?
    • pphysch 9 days ago
      This is the kind of thing that is totally technically plausible, and useful to intelligence groups but not the general public, and therefore I would assume it is 100% in production in international intelligence agencies. But nothing we can "prove" without getting shot.

      Similar to ultra-accurate multimedia geolocation models.

    • tacostakohashi 9 days ago
      Things like it have happened. The Unabomber was identified through his writing idiosyncrasies.

      Lots of criminals have been identified by reanalyzing old DNA samples using data and genealogical techniques that weren't possible at the time the samples were left.

      • ButlerianJihad 9 days ago
        > The Unabomber was identified through his writing idiosyncrasies.

        By his brother, apparently. His brother read the manifesto, and said, “hey, sounds eerily familiar!”

        But wouldn’t it be deliciously ironic if A.I. had caught Kaczynski‽

      • iammjm 7 days ago
        Unabomber is a very specific example. He was an outspoken dude with some peculiar convictions which he liked to share with people. I don’t think it makes sense to compare him with all us normie shitposters
    • axus 9 days ago
      As mentioned in a comment above: https://news.ycombinator.com/item?id=33755016
  • sixtyj 9 days ago
    > A stronger conjecture is that we’re heading towards a sort of generalized pseudpocalypse. Perhaps, in the future, if you interact with the world through essentially any high-bandwidth channel, then you identify yourself. Say you wear a mask in public and only speak by sub-vocalizing into a voice changer. That’s fine, you’ll still be identified using your body shape, gait, or chemical signature. Or say you don’t like your car being tracked everywhere, so you stop carrying a phone and you somehow convince lawmakers to ban license plates. No problem, your car will still be tracked using tiny scratches or unique pinging sounds from the engine. Or say you don’t like being tracked on the internet, so you lock down your browser profile, buy stuff only with Monero, and connect through a chain of three VPNs. That’s OK. You’ll still be identified through how you wiggle your finger as you scroll down the page. We’re all just too unique, and the information theoretic limit is coming for us.

    Forensic research, NSA, Palantir…

    Btw 42. Sleep, eat, have sex, have fun, be useful.

    • inigyou 9 days ago
      Useful to whom?

      See, I used "whom", messing up my fingerprint.

      • sixtyj 9 days ago
        Just be useful. To community, to family, to company, to yourself.

        Sometimes I see people that consume and take only. (I am not judging, I just observe.)

  • losvedir 9 days ago
    Huh, I just realized he (she?) was pseudonymous and not Matt Might[0] this whole time. Oops.

    Also, I wonder about this analysis in the age of AI slop. I wonder how much that removes the identifying bits, vs how much carries through of the original prompt (e.g. topic and guidance). It's interesting that a pseudonymous blogs might take on very generic Claude-voice, which could be worthwhile if the topics were interesting, but could also just be a completely humanless bot.

    [0] https://matt.might.net

    • hungryhobbit 9 days ago
      AI is clearly the correct answer to this specific problem: Claude can not identify another writer if they use Claude (or another LLM) enough to massage the text.

      The problem, as the article correctly identifies, is that this is just the tip of the iceberg. There are tons of other cases (e.g. identifying a car without a license plate by the scratches) that AI is going to enable ... and in those cases we can't just get anonymity back by using an LLM.

  • tokai 9 days ago
    Eh isn't authorship linking a whole field of study? Yes it is, called stylometry. Here's a review paper form 2006 [0]. Its an old subject, with literally thousands of papers. Really wished the author had taken a cursory glance at the literature.

    [0] https://link.springer.com/chapter/10.1007/11889342_20

    • Smaug123 9 days ago
      The article contains Dynomight’s thoughts on the literature of stylometry. Search on the word “stylometry”.
  • fullstackchris 9 days ago
    a lot of these comments have no understanding of basic statistics. for example, did you know, at just 30 words of grammatically acceptable text, at least in english, youre already at more possible combinations of phrasings than atoms in the universe. you cant just throw a problem like that at LLMs and expect them to "just work"

    folks should also look at burrows delta - i forgot which books but some folks were able to identify a ghost writer by stylometry alone.

    ive said this on many threads, you cant just have text output from an llm (regardless of style / "pseudonym") and have it be "unique" because the nature of the transformer model itself is literally present in the output words. it will be detected as llm output every time. it has to be!

    for pure anonymity, i suppose then it is an answer... for the actual craft and art style of writing it is not

  • thomassmith65 9 days ago
    The best mitigation a person has is to have an LLM 'flood the zone' with slop based on her writing style.

    So she has one comment on the internet admitting she cheated on her taxes, another copping to an axe murder, another revealing she's the one and only D B Cooper.

  • altcognito 11 days ago
    [flagged]
  • alexwebb2 9 days ago
    [dead]
  • cestith 9 days ago
    One potential solution here I suppose is to make deanonymization or contributing to it a serious crime. I doubt that will happen in most places, though, since it’s often the government that wants to do this to its own citizens.
    • inigyou 9 days ago
      downvoted because the government doing stuff is communism
      • tarpitt 9 days ago
        I don't have the ability to downvote, and I disagree that the downvote should be used, but I disagree with your disagreement about the downvote.

        The proposal that seems to infringe on free speech, but mostly I take issue because there just isn't really a precedent for what such a thing would look like. What constitutes doxxing? It is easy to accidentally doxx someone by mentioning some offhand fact in conversation, because you may not know what series of facts can be connected to form a doxx. Typicially, people doxx themselves by accident, and someone else is merely pointing it out. I know I have many times. You live and learn.

        There isn't really a right to privacy in the way that there's a right to free speech. Free speech is important to processes of public transparency and justice which I think this would interfere with. But it's also true that justice is blind and doxxing can interfere with a judicial process.

        Also, if everyone is going to have access to super-stylometry tools in the future, which is the premise of the article that GP reacts to, it will be fruitless to uphold such a right because anyone can just run doxxyou.exe themselves. There's no need to spread doxx because it can be reproduced individually.

        The term doxxing came from a certain hacker culture where it was implied that you connected some real-life identity to a criminal pseudonym. It essentially meant "snitching". So the idea that doxxing itself would be a crime is interesting. It's a reversal of the original meaning.

        But it's also true that the public has discovered sybil-suseptible techniques (like swatting) where you can screw with ordinary people's lives by knowing their identity. Which is interesting because we live in a new culture of "share everything online" vs old the hacker ethos of "don't use your real name online". The attackers have become stronger and the defenders weaker.

        • cestith 8 days ago
          Free speech repeatedly has been held by the US Supreme Court to include anonymous speech. There are also free association rights involved. https://firstamendment.mtsu.edu/article/anonymous-speech/

          While the protection of anonymity is not absolute and must be balanced against other factors, the same is true of free speech in general. You don’t have a right to defraud people, defame them, directly incite a riot, or to manipulate markets with false statements for example. You likely don’t have a right to create a deepfake and represent it as the words and actions of another person without their permission.

          Deanonymization can be used for harassment, stalking, prosecution, ad targeting, serving civil lawsuits, criminal investigation, and any number of other purposes both benign and nefarious. It’s honestly akin to stalking itself when it’s done to a private citizen by a private citizen.

        • inigyou 9 days ago
          probably means publicly identifying an individual
          • tarpitt 9 days ago
            It takes many bits to identify an individual.

            Let's say I offhandedly mention your email in this comment, for example, like we had an out of band conversation somewhere else. Then, someone takes that email and links it to another forum where they find your phone number. Then another person looks up that phone number in a phone book. Then another person finds your github account from the email associated with your PGP and finds where you work etc. Let's say a comment you wrote 7 months ago vaguely mentions something about where you live, which in combination with the previous information narrows it down to a single place.

            At what point does that become doxxing and who is responsible? Pseudonymous people nessisarially, slowly leak small amounts of information about themselves when on the internet in order to engage in communication.

            It is only when you collect "enough" bits of information that it becomes threatening doxx.

            You could say publishing this collection of facts is the doxx. But the point of the article is that stylometry and AI tools can do this investigation for you. Anyone can trivially assemble the collection themselves.

            I think ultimately, it's a matter of personal responsibility. It has to be. If you have perfect opsec, I can't touch you AI tools or not. And if I have perfect opsec and I'm doxxing you, then no laws will be able to catch up to me.

            • inigyou 8 days ago
              Laws aren't cryptography. They aren't all or nothing. Making something illegal has a chilling effect on that thing, even if not 100% enforceable. It would be illegal to try to hack the NSA, so I'm scared to do it, even through Tor.
              • inigyou 8 days ago
                ... And even though it's hard to define whether any given action is or isn't hacking.
            • cestith 8 days ago
              The article is about plugging in one or more pieces of writing and getting a name. That’s not leaking their phone number, their email address, their location, their associations, or other information and piecing together an investigative file on someone. It means that with AI, based on the idiosyncratic writing style of “tarpitt” on HackerNews, I could get your real name no matter the topics you commented about. It assumes only that you have other writings available somewhere that can be tied to your real name.
      • cestith 8 days ago