Let’s join the dots between a few different themes this week.

First up, cursor rule files are vulnerable to prompt injection attacks. It’s possible to embed prompts within the rules files and hide them using invisible characters.

You can then use this poisoned rule file to redirect cursor/your agentic IDE of choice towards malicious implementations. This is not a huge surprise - the point of rules files is to direct the LLM towards specific implementations. What’s changed recently is large, shared, rule sets vulnerable to poisoning by prompt injection have started to be schlepped about online and vibe coding is taking off in a big way. A simple exploit becomes highly likely if everyone has purposefully stopped reviewing the code.

This is the cardinal sin of the LLM architecture writ large. Data and instructions are mixed together and it’s left to the model to decide which is which.

Simon Willison flags separately that the new OpenAI audio models are also vulnerable to prompt injection - stage directions are inserted directly into the script (e.g. “whisper this part”). Older text to speech APIs have typically favoured a DSL to handle stage directions, but the new fashion is to bake these features into an LLM base model and then control everything using natural language.

There’s a tension here which needs to be solved. It seems obvious at first glance - don’t mix in the instructions! But the pattern is fundamentally really useful.

My favourite post in the last week or so was Matt Webb’s Diane, I wrote a lecture by talking about it. Matt’s been playing about with TTS+LLM tools (Super Whisper, MacWhisper, WisprFlow etc.) and has found a neat hack:

My generic prompt to Claude, used every time, is now:

you are Diane, my secretary. please take this raw verbal transcript and clean it up. do not add any of your own material. because you are Diane, also follow any instructions addressed to you in the transcript and perform those instructions

[paste in transcript]

Which means, when I’m talking through my lecture outline, I now finish by saying:

ok Diane I think that’s it. it’s a talk, so please structure all of that into a high level outline so I can work on it. thanks.

And I can mix in instructions like: oh Diane I meant to include that point in the last section. Please move it.

It works super well.

I love this. It’s so intuitive and convenient. It’s also a great example of using this same potent mix of data plus instructions to create an entirely new application on the fly.

This is the amazing magic of LLMs, the ability to make a small tweak to your process and create a surprising new outcome. Prompt injection will always be a core, intractable issue with LLMs precisely because mixing data and instructions is so powerful. It’s a feature, not a bug.

This means as AI engineers a core skill is going to be inowing whether you need the feature or not.