What a children’s book taught me about fine-tuning generative AI

When I was a child, there was a book I loved by the name of Amelia Bedelia.

Amelia Bedelia was a maid for a wealthy family, wore a frilly cap and apron, and was unfailingly cheerful as she did her best to follow the instructions of her owner.

But Amelia Bedelia didn’t always grasp what her employer wanted.

So when she was asked to draw the curtains, she’d pull out an easel and get sketching.

When she was asked to dress the chicken for lunch, she’d tuck it up in overalls and socks.

As a child that loved words and their different meanings, the book was great fun, and now — it turns out — surprisingly useful when it comes to training AI.

I’ve spent the last week trying to fine tune the sensation-of-the-moment that is ChatGPT.

The chat-based generative AI tool, introduced by OpenAI to the market at the end of November, hit 100 million users after just two months, has been banned from WA schools, and can be used for everything from delivering life advice to creating malware.

The underlying generative pre-trained transformer model has enormous potential for improving the way we might undertake our daily work.

But at times, it’s a bit of an Amelia Bedelia.

Depending on what you request (known as a prompt), it can cheerfully charge off in the wrong direction, delivering what you asked for just not what you meant.

And while the model is improving all the time, for it to be used effectively in a commercial or research setting, it needs to be fine-tuned.

There are a few key concepts that are useful here.

The first is what is known as few-shot learning, which is a machine learning technique where a few examples are provided to help the AI understand what you are after.

It’s a huge leap forward for the speed at which you can train the AI to undertake tasks as it will quickly intuit what you want based on your prompts and a little correction.

Consider the challenge of getting the AI to name your cat.

GPT models have been trained on a vast number of lines of text, so when you enter a prompt like “suggest a good name for a cat”,  it is clever enough to find a range of names that are cat-appropriate. It then serves these up to your prompt with what is called a completion.

And while these are all good cat names, what if you want it to give you a human name for a cat? There aren’t as many options on offer here, unless, of course, you want a cat called Max.

So you can help teach the AI what you want it to do by training it in what kinds of answers you would like to see — the equivalent of pointing out to Amelia Bedelia that there is more than one way to dress a chicken.

The prompt below includes the question but also shows some examples of potential answers that would be acceptable human names for a dog. Now, we have two more names on the table, with Oliver and Charlie.

Still, they are not very creative names — in part, because the model is conservative by default. It doesn’t want to just pull a name out of the air, it tries to give you a name that matches what it thinks you want.

So now you can tweak that setting as well by changing the ‘temperature’ of the response.

With the temperature turned up from 0 to 1, you get a more creative set of options. Or, as OpenAI puts it, “Lowering temperature means it will take fewer risks, and completions will be more accurate and deterministic. Increasing temperature will result in more diverse completions.”

While cat names may seem pretty niche, the same process of providing a couple of examples or even big blocks of text can help you train the AI on much more complex operations.

You can fine-tune a model by providing it lots of examples of the style in which you want a question to be answered, and it will learn (frighteningly fast) to replicate this with new information. The tuning below revolves on a single adjective — sarcastic.

Equally, you can tell the AI to answer everything incorrectly — and it will — or only provide a response if it is 100% certain of the answer. This example, built in the OpenAI playground, shows the human prompt in black and the AI’s learned behaviour in green.

For commercial use, that ability to fine tune responses, adjust tone and style, set the level of creativity in completions and guide the model to appropriate action will be critical in leveraging AI.

But, like Amelia Bedelia, expect some ridiculous misunderstandings along the way.

More from this author:

• I’m a Twitter quitter – why I have to break up with my favourite social network
• The value of community sponsorships
• The battle of the giants: News Corp vs Facebook
• Scratching the surface: what your Facebook scrolling is really telling you