I fine-tuned a small local model, Qwen3-4B, on about 17,600 of my own X posts and replies, most of them paired with the post I was answering. It runs on my MacBook, and when two LLM judges had to guess which of four replies was really mine, the model was their favourite wrong answer. It also regularly writes complete nonsense.
This post is the method, step by step, so you can fine-tune a local LLM to write like you. I used my X archive, so that's the worked example. The same recipe should work for any writing where you reply to something: Reddit comments, email threads, Slack. I've only done it with X.
How I tested whether it really sounds like me, and what fooled me along the way, is in the companion post: Does your fine-tuned LLM sound like you? How to test it.

What you need
- Your writing, ideally in reply form. I used my X archive (Settings β Your account β Download an archive of your data). It takes a day or two to arrive.
- A Mac with Apple Silicon and 16 GB of RAM or more. I used an M3 Pro with 36 GB, and training peaked at 6.1 GB.
- Python and mlx-lm, Apple's library for running and fine-tuning models on the Mac.
- A few thousand posts. I started with 12,985 phone posts after cleaning and later added about 6,600 desktop replies. I haven't tested how few would still work.
No GPU server, no API bill for training. The only paid parts were optional: writing the briefs for my first version, and other LLMs as judges. Both cost me little.
Step 1: export your writing
For X, the archive has a file called data/tweets.js. It's JSON with a JavaScript assignment in front, so strip everything before the first [:
import json
s = open('tweets.js').read()
tweets = [x['tweet'] for x in json.loads(s[s.index('['):])]
Each tweet has full_text, created_at, source (which app you posted from) and, for replies, in_reply_to_status_id_str. That last field is the most important one in this whole project. More on it in step 3.
Then clean:
- Drop retweets and duplicates.
- Decide about links-only posts and very short ones. I kept replies of 20+ characters for testing.
- Decide about languages. I write in English and Polish. My first model mixed both and sometimes answered English posts in Polish, so I trained the later ones on English only.
Split by date, not at random. My test set is the latest 400 posts. A random split leaks: if I post about the same thing three times in a week, a random split puts two in training and one in test, and the model "remembers" instead of writing. Validation is a random 5% of the rest.
Step 2: the input matters more than the output
A reply alone is useless as training data. The model needs an input to learn what you do with it.
My first version made the input up. I had Claude Haiku read each reply and write a one-line brief backwards from it, which is called instruction backtranslation:
Brief: "Reply to someone who misread your project's description. Thank them and clarify you meant the license."
Reply: "Hah, Thank you! I meant the license π"
The model trained on that looked great in my tests. Then I noticed the brief already contains the answer. "Thank them and clarify you meant the license" is the reply minus the typing. In real life nobody hands me a brief. I see a post and react to it.
When I switched every test to the real post, the brief-trained model collapsed: 73-5 for Claude in pairwise ranking. It mostly echoed the post back.
So train on what you actually see: the post you replied to β your reply.
Step 3: fetch the posts you replied to
Your archive has your replies but not the posts you answered. It does have their IDs (in_reply_to_status_id_str).
I fetched the text of each parent post from the endpoint X's embedded tweets use. It's undocumented, so it can break any day, but it needs no API key:
https://cdn.syndication.twimg.com/tweet-result?id=<tweet id>&token=a
One request per second, saved to a local file. That's 22,886 posts, and 1,066 came back empty (deleted or protected). In the final set, 544 replies whose parent I couldn't fetch kept the made-up brief from my first version, and so did 604 standalone posts, which have no parent at all. Desktop replies without a parent I simply skipped, and I'd do that for everything next time.
Then each training row becomes a chat in the format mlx-lm expects:
{"messages": [
{"role": "system", "content": "You are Marcin Dudek. Write the message yourself, in your own voice, as you would type it on your phone."},
{"role": "user", "content": "Reply to this X post:\n\n<the parent post>"},
{"role": "assistant", "content": "<my real reply>"}
]}
Keep your reply as typed. I stripped links, leading @mentions and HTML entities, and kept everything else. Typos, missing apostrophes, ;) instead of an emoji: all of that is part of how you sound. My first instinct was to clean it. Don't.
One gotcha: I first used phone posts only, because I had a vague worry that some desktop posts were AI-assisted. They weren't, and later I checked properly. I had independent readers write blind portraits from phone replies and desktop replies separately. The two came out the same person, except that desktop replies are longer. So I added the desktop replies too: 17,614 training rows, 16,466 of them with the real parent post.

If you're not on X, look for the same pair in your export. I haven't tried these, but each one records what you replied to: Reddit's data export lists every comment with its parent, a Slack export keeps thread IDs, and an email thread quotes the message you answered.
Step 4: train the LoRA
A LoRA trains a small set of extra weights on top of a frozen base model. The base stays the same, and my adapter is a 59 MB file.
I used mlx-community/Qwen3-4B-Instruct-2507-4bit. This is my config, minus the logging settings:
model: mlx-community/Qwen3-4B-Instruct-2507-4bit
train: true
data: data/mlx_parent_all # train.jsonl + valid.jsonl in the chat format above
fine_tune_type: lora
mask_prompt: true # learn only the reply, not the post
num_layers: 16
batch_size: 8
iters: 400
learning_rate: 1.0e-4
max_seq_length: 512
grad_checkpoint: true # without it, memory passed 16 GB in the first batches
adapter_path: adapters_all
seed: 42
lora_parameters:
rank: 16
scale: 20.0
dropout: 0.05
keys: ["self_attn.q_proj", "self_attn.k_proj", "self_attn.v_proj", "self_attn.o_proj",
"mlp.gate_proj", "mlp.up_proj", "mlp.down_proj"]
pip install mlx-lm mlx_lm.lora --config lora_config_all.yaml
The first run downloads the base model, about 2.3 GB.
On my M3 Pro that's about 45 minutes. Peak memory was 6.1 GB.
Two settings I'd call important:
mask_prompt: true. Without it, the model also learns to produce the posts you reply to. You want it to learn only your side.grad_checkpoint: true. It trades some speed for a lot of memory.
One more memory gotcha. One of my runs died at iteration 325 even though the process was using only 2.2 GB. The growth was MLX's cache of freed buffers, not real tensors. Capping it fixed it without touching any training setting:
import mlx.core as mx from mlx_lm import lora mx.set_cache_limit(2 * 1024**3) mx.set_memory_limit(12 * 1024**3) lora.main() # same argv as mlx_lm.lora

The validation loss stops moving after about 100 steps, so past that point it tells you little. The blind test in step 6 is what tells you whether it worked.
Step 5: generate
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
model, tok = load('mlx-community/Qwen3-4B-Instruct-2507-4bit', adapter_path='adapters_all')
msgs = [{"role": "system", "content": "You are Marcin Dudek. Write the message yourself, in your own voice, as you would type it on your phone."},
{"role": "user", "content": "Reply to this X post:\n\nWhat's your favorite open weight model?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt=prompt, max_tokens=220, sampler=make_sampler(temp=0.7, top_p=0.9)))
Put your own name in the system prompt, and use the same system prompt and user-turn format at generation as in training. If you change the wording, you're asking the model something it never saw.
What it writes: six real examples
All six are from held-out posts the model never trained on. "Me" is what I actually replied. "LoRA" is the model's reply to the same post. I picked them to show the range, from good to garbage, not only the best ones.
Post: a news story about a 97-year-old man who went viral after dying at his own birthday party, dancing to "Stayin' Alive".
Me: GONE viral for good π RIP Vito
LoRA: Some people are so old that they can't remember the Bee Gees π
Post: a joke that taxes depend on which side of them you're on.
Me: There's only one side if you're doing serious business π
LoRA: On the other side of the tax, but I'm still feeling really generous today :D
Post: someone sighing that they now have to file a lawsuit against a person.
Me: *hey Claude, new fun task for you!*
LoRA: You just wasted me $200 π
Post: someone looking for a way to let their AI agents, running in different places, talk to each other.
Me: Don't build it. It's a security nightmare. Imagine someone finds it and works hard to tell your agent to give away all of your passwords, credit cards or whatever he has access toβ¦
LoRA: I have a solution for that! Seriously!
Post: someone with four app ideas who can't decide which one to build first.
Me: Give each one of them to an agent to build. We don't really have to choose anymore
LoRA: You know what to do, just build.
Post: excitement that Kimi K3.1 scored right behind Opus 5.5 on Terminal Bench 4.0.
Me: Is it available in Devin? π
LoRA: So I'm wondering - if we can go to the Astra and plot Astra - 2 - 1 - - 3 - 5 is the "best" path to Terminal Bench 4.0
The pattern is clear once you read a hundred of these. The shape is mine: the length, the π , the dry one-liner, " - " for asides, ":D". The content is often off. A 4B model frequently doesn't understand what it's replying to, so it produces something that sounds like me reacting to a different post. On the agents post I gave an actual opinion with a reason. The model gave the tone of an opinion with nothing in it.
Step 6: test it blind, or you'll fool yourself
This is the step I'd spend the most time on. I ran about a dozen test rounds, and the leader changed as I fixed my own tests.
What finally worked:
- Hold out 100 real replies with their real parent posts. Never trained on.
- For each post, mix your real reply with the candidates (your LoRA, a prompted frontier model, other LoRA versions), and shuffle them.
- Give a judge 20 of your real posts as reference, then ask it to rank the candidates and guess which one is really you. I used Grok and Devin. I left Claude models off the panel when a Claude model was one of the candidates.
- Score two ways. "Picked as the real one" and pairwise ranking points. They can tell different stories from the same judgments.
On the main run (the first parent-post model, phone replies only), the judges found my real reply 157 times out of 200. Of the 43 misses, 30 went to the LoRA, 8 to Claude Haiku with my style prompt and 5 to Claude Sonnet with the same prompt.

Then measure your noise. I judged the same 100 replies three times, changing only the judge's shuffle and example posts. One pair scored 118-82, 101-99 and 111-89. So on 200 judgments, a single run can't separate anything within about Β±20. Run it at least three times and look at whether the direction holds. Most of my "improvements" after the first working model were inside that band. The companion post has the full numbers.
Surface vs character
The judge's instructions decide what "sounds like you" means. My first judge prompt said "wording, length, punctuation, emoji habits, rhythm". Those are surface. The LoRA nails them: on the 100 test posts it used an emoji in 41% of replies, exactly like me. Sonnet with a prompt that says "about one in three" used one in 56%.
But how I sound is mostly what I do in a reply: ask how something works instead of saying congrats, push back with one counterexample, joke at governments and hype instead of at people. To judge that, I had four readers go through 3,000 of my posts and write a portrait of my character. Then I gave that portrait to the judges as the rubric, and told them to ignore length, emoji and punctuation.
On character, my models came out within a few points of each other, and the judges picked my real reply 158 to 168 times out of 200. The LoRA copies how my replies look much better than how I think.
What didn't help
- Averaging LoRAs ("soups"). Two runs with different seeds learned weight changes that pointed in mostly different directions, yet scored the same. Stacking them into one adapter gave a small win in one round and nothing I could separate from noise later.
- Forcing my emoji rate with a logit bias. It matched the number and moved the judges by nothing.
- A length hint ("Length: short | medium | long" in the prompt). Told the length of my real reply, the model hit the right length 76-82% of the time, against 29-41% before. The judges barely moved.
- Activation steering. I took the difference between the model's internal activations on my real replies and on AI imitations, and pushed generation along it. At one strength the model started answering in Polish. Pushed harder, it announced "I am Marcin Dudek, and I am a man who lives in Poland." It never started asking questions the way I do. That direction captured who I am and where I'm from, not how I react.
Where it breaks
It only knows X replies. I asked it for an email and got a tweet. A blog intro came out generic. If you want a model that writes like you everywhere, you need training data from everywhere: emails, posts, docs.
It doesn't know what it's talking about. When I picked the better reply myself, blind, Claude with my style prompt won 23 to 3. It understands the post. The 4B model mostly doesn't.
Judges are noisy, and so is one person's opinion of their own voice. I'm the only human who rated anything here. The next proper test is 3-5 people who read my posts, guessing which reply is mine.
Would I do it again?
Yes, but for what it taught me, not for the model. The part I'd keep is the data prep (real parent posts, a date split, typos kept) and the blind "which one is real" test. The most useful thing I got out of the project is the written portrait of how I react on X. It came out of trying to define what the judges should look for.
If you build your own, start with the test. Decide what "sounds like me" means and how you'll measure it before you train anything. I did it the other way round and spent three rounds measuring the wrong thing.
Published · Last updated