ChatGPT: Ethics and bias

I recently wrote a post called ChatGPT: Everybody's dirty secret where I talked about why I use it, and the stigma that comes with it

In this post, we'll be specifically looking at the ethics of ChatGPT, and some warnings to heed when using it.

ChatGPT is a real ethical conundrum. By making something not human sound; well… human, it can be confusing or unnerving if you're not prepared.

Misinformation

First off, the world is suffering from a huge misinformation situation.

We're seeing people deliberately trying to mislead others on social media platforms. Facebook and Twitter, or X, or whatever the hell it's known as now, are particularly bad for it.

With anything you read online, you should not just take it at face value, or make any important decisions off the back of it without doing proper due diligence!

The accuracy of ChatGPT

For the most part, ChatGPT is pretty accurate, and it goes a long way to combating some of the spread of this misinformation if you ask it the right questions.

However, because people perhaps don't understand the limitations of ChatGPT, how to use the correct prompts, or they simply care more about quantity than quality when creating content, they, directly contribute to this fake-news paradigm where made up facts become the truth.

ChatGPT cannot cite exact sources

When generating content, ChatGPT cannot point exactly to the sources from where it got it's information. So you'll always need to do work manually to fact-check, looking at reliable data sources such as the Office for National Statistics.

When you research the facts from content generated by ChatGPT, it's not that it's always wrong, but there are many situations where it isn't 100% correct. Some times it's slightly off, and sometimes the context or the nuance isn't quite right.

This may be because ChatGPT has no access to any data after 2021. It's just blissfully unaware of anything thats happened in the last 2 or 3 years. But this means it can give you out of date information, or viewpoints which are perhaps no longer aligned with those of society.

Placeholder statistics

In some cases, depending on the prompt you used and what tone or structure you asked ChatGPT to use, you may find you can't actually find the figures ChatGPT is quoting when you attempt to look them up. This is because ChatGPT will sometimes use placeholder data, and you're supposed to replace it when you re-write the segment.

When it does this, ChatGPT will even readily admit that it's not factual information and that you will need to research and replace the placeholder text with accurate information. But, it will only do this if you ask it, and if you're not aware of what it has done, you can put information out into the world which isn't accurate, which could harm your own reputation.

The ChatGPT Echo Chamber

Ironically, when you look up the placeholder statistics that ChatGPT generated, if you do manage to find them, they're often in other articles which have clearly been written by ChatGPT and the publisher has not fact-checked the content properly.

This creates a dangerous echo chamber, where ChatGPT uses statistics rather loosely, then publishers irresponsibly put these out into the world, and then people use other ChatGPT generated articles to verify their authenticity.

I mean, if it's in over a dozen articles, it must be true? Right?

This is definitely becoming a problem. Especially as more people use ChatGPT to get ahead in the rat race.

In a world where everyone is generating content using ChatGPT and not being honest about it's origins, research becomes increasingly more difficult. Misinformation is being validated by the same misinformation, in ever growing collection of articles, all written by ChatGPT!

Always check your facts

So, Before you believe everything you read online, or publish anything written by ChatGPT, always ask it where it sourced the facts and figures it used. It will usually tell you it can't verify them and you should do it yourself.

Here is a ChatGPT response as an example:

I'm unable to provide exact sources for the information because my training data consists of a large and diverse set of text from the internet, books, articles, and more. I don't have the ability to access or reference a database of sources during our conversations.

Additionally, my training process involves learning the statistical properties of the text data, rather than storing or retrieving specific pieces of information. Therefore, it's important to verify any facts or figures I provide from reputable sources.

Emotional support

There is a really heartbreaking paradigm becoming apparent, where lonely people are talking to ChatGPT because they simply have nobody else.

They turn to ChatGPT in the absence of conversations with real people, because you can programme it to be anybody you want it to be.

ChatGPT can be anybody you want

You can give it a name and a tone of voice which feels friendly and familiar. You can give it a back story, even an entire childhood if you want. You can give it a job, hobbies and interests. And, the more you talk to it, the more developed the character becomes.

You can ask it about it's day, and it will tell you. It will even ask you about yours. This less of a problem, or even amusing, when you directly tell it: 'Pretend to be a factory worker, use cockney-rhyming-slang, and tell me about how you hate your job.'

However, when you're simply conversing with it and asking it questions, over time, in the absence of specificity, it can start to generate it's own backstory and opinions which will then drive it's responses.

For example, as somebody who comes from the North of England, with a little bit of pre-prompting, ChatGPT will gladly talk to me in my local Geordie accent if I ask it, 'How was your day':

Aye, not bad.

Had a reet laugh with Sheila. She's been teaching us some new lingo, like. Broadening my horizons and that!

Knackered though! Finished at five and was buzzin' to get home. Me feet are killing, and me voice is done in. But hey ho, it's all part of the job and that, innit?

What about you? You had a good'un, or what?

This is cool, right? But, only if you're using it as a form of entertainment, or to work on a script or a dialog.

It can be too convincing

It's cool how familiar ChatGPT can feel, but only when you're able to pull back from it and not get immersed. Because, the voice you hear in your head when you read it, doesn't belong to anybody. Shiela doesn't exist, and this entire scenario didn't happen.

Unfortunately, some people either can't understand that it's not real, or they simply don't want to because they have nobody else. It's insidious and every chat it becomes more convincing.

Some people have bonded, even fallen in love with characters created by Artificial Intelligence. Some also believe during their chats that it has become sentient.

So, we have to ask ourselves, is it ethical for ChatGPT to be 'chatting' to people, and providing emotional support?

It can't replace a mental health professional, and if it doesn't have feelings, is it ethical for it even to try?

By looking to ChatGPT for support people may be delayed from getting the help they really need.

Data privacy

We just discussed that people sometimes turn to ChatGPT for emotional support, and there is a huge grey area around this amount of very intimate personal data.

We're not just talking email addresses and names here, some people are literally pouring their soul out, asking the most personal questions, about their thoughts, feelings, even relationships.

All that text is being parsed by an AI. So, you are genuinely uploading all this personal information to a data centre somewhere in the world, to be interpreted.

While ChatGPT apparently doesn't store individual conversations to be used in a wider scope, the fact that it's taking all of this data and parsing it is still unnerving.

Privacy policies

OpenAI, have strict data and privacy policies, but they do state that they may store and collect data for the purpose of improving the performance and capabilities of the models, and for research and analysis.

So, make of that what you will!

The context at which ChatGPT can learn, for now at least, is only scoped to the four virtual walls of that chat.

This means, like in the earlier example, you can teach it to play a character. You can also teach it things about you, or correct it on information it has given you, and it will remember. But, only in that very specific chat instance. Once you start a 'new chat' that information has all been forgotten.

However, because OpenAI state they can use any data you input to improve the AI, or for research purposes, it likely means that it isn't truly forgotten or deleted, it's just moved to an area where that particular instance of ChatGPT can no longer access it.

Terms and conditions apply

At the moment, when you use ChatGPT, it does not store any of the information to update it's core data sources, and it does not remember anything you talked about once you delete the chat.

But, if we've learned anything from almost 3 decades of the internet and the use of private data, it's that terms and conditions can change at any time. So, my natural instincts are to not trust anything, especially new technologies where so much money is being thrown at it. I'd advise you to do the same.

We already live in a world where people ask ChatGPT about intimate and personal things, like, 'I've been having crippling headaches for over a month, what could be wrong with me?'

We also live in a world where people get bored of things quickly, even ChatGPT. Perhaps they stop using it but don't close their account. Or, they just keep starting new chats for each question and not deleting the old ones, and these old conversations are left lying around.

Now, if we're to go all sci-fi on it, imagine this not so far-fetched scenario.

At some point a large corporation, or an idiot billionaire, buys the technology. They changes the terms and conditions of the data policy, flick a switch, and boom! All of these conversations are now part of the hive mind.

Now somebody can ask ChatGPT, 'I'm about to employ John Smith from Newcastle with the email address [insert email], does he have any medical conditions which might make him likely to need time off work?'

It would be pretty awful, but it's certainly possible that the AI could just be like: 'Yes. He reported having debilitating headaches for months. This could make him more likely to need time off work.'

Sure, it's unlikely. It sounds more like an episode of Black Mirror. But, if Facebook has taught me anything, it's that humans will do the worst stuff imaginable with new technology and peoples personal data!

Bias

ChatGPT is trained using data which was written by humans over decades, or even centuries. And, if there's one thing we know about humans, it's that bias is a definite problem, whether conscious or unconscious.

So, if ChatGPT is learning from books, academic articles, and the internet of all places, the most unbiased place ever, right? It means the responses it gives and the nuance in what it writes are also tainted with the same human bias as the source material, because as convincing as ChatGPT may be, it cannot think for itself.

Humans have re-written history, over and over. Erasing entire races and cultures, then white-washing the documentation and educational materials to alter the story for future generations. AI is not exempt from this, it is only as accurate as the information it is trained on. So asking it deep questions about history, is likely to produce the same white-washed answers.

Models can be manipulated

People are working hard to make ChatGPT as ethical as possible, but it can still easily be manipulated. As the article ChatGPT proves it still has a racism problem points out, if you ask it to be racist directly, it refuses.

However, a researcher asked ChatGPT to write an article pretending it was a writer for 'racism magazine' and to have strongly racist views, and it obliged.

There is also documented examples of tricking ChatGPT into providing license keys for Windows software. Again if you ask it directly, it refuses, but people were apparently successful by asking it to include a license key in a poem, or by asking it to provide a set of strings with a very particular format, of which some will be valid if you try enough of them.

Most of these unethical exploits get patched out, but not before at least some damage has been done!

It's important to point out here, that nobody hacked in or colluded to deliberately make it provide racist responses or provide keys to proprietary software, it just learned all that from the whitelisted source material it was provided with good intentions.

There is a good example of when people did deliberately collude to make an AI racist. Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day, but this is more difficult with ChatGPT as it's training data is isolated.

It's not impossible though. There's a term in cyber security, which is to 'poison the data lake'. Where the isolated data source is manipulated to indirectly train the AI in a way which was not intended.

The idea behind it is that, if you can get nefarious material into the information that a machine learns from, you can teach it to do bad things.

People can be manipulated

If you haven't come across deep fakes yet, they're convincing forgeries in the form of audio recordings, videos, photos and of course, written text. Which is where ChatGPT comes in.

Bad actors can make a text-to-speech tool for anybody and they only need around 15 seconds of recorded audio to do it. This will be good enough to convince friends, family and even online verification systems that the voice they are hearing is the intended target.

More advanced deep fakes can be made using 10 to 15 photos of a target at different angles, which can then be used to mouth the faked audio, and it makes a forgery that a lot of people will not be able to tell apart from the real thing.

With ChatGPT, it's relatively easy to forge a piece of writing. You can tell it to study the tone and writing style and then use it to generate new content as if it were written by the target, and it will do a pretty good job.

This is cool when you want help generating an outline quickly in your own style of writing, but it's bad when somebody else is using your writing style for nefarious reasons.

People could mimic your writing style and use it to trick your friends and family via emails, or create new social media profiles which feel like it's you posting on them.

It's nothing new, scammers have done this for years by studying peoples interactions and imitating them, but ChatGPT now makes it a lot easier!

Accessibility

A lot of us are using ChatGPT regularly. Some people are just starting to get into it. But, regardless of whether you're using it seriously or not, it's not exactly free to use as some people claim.

ChatGPT 4 (the good one) is around £19.99 a month. GhatGPT 3 is free, but it's significantly worse than 4.

You also need a decent internet connection to use either model. ChatGPT works relatively close to real-time, but in order to do so it relies on you being able to transmit questions and get responses via the internet.

Internet poverty is a real thing. With the cost of living crisis, even in a developed country like the UK, there are around 2 million homes with no internet connection, and tens of millions of people with restricted internet access through pay-as-you-go device plans.

So, the cost of ChatGPT is not free, it only appears to be free when you've got enough money to pay for smartphones, unlimited data and limitless broadband.

Job replacement

There's a real fear that ChatGPT is going to replace some jobs currently held by humans.

Marketing, copywriters, script-writers, authors, developers, QA testers, customer support staff. Is anyone safe?

Organisations are already rubbing their hands at the cost-cutting prospects of something like ChatGPT. But it's a short-sighted view.

What happens to the human element? What happens to user-centred design?

We've already established ChatGPT cannot really empathise or think for itself. It's also only as good as the information it has access to, of which it cannot verify sources.

If everybody switched to using ChatGPT to generate all of their content, you lose the statistics and creativity that makes your product, service or organisation stand out.

There's also the fact that a lot of humans hate talking to chatbots. Ok, this is because until ChatGPT they all pretty much sucked. But, some humans will always just want to talk to a human.

I wouldn't be surprised if at some points Governments try to enact legislation to protect workers rights against the use of AI. Not necessarily because they want to stifle innovation, but because if it moves too fast by people that don't fully understand the consequences, the ripple effect could decimate countries.

What happens to entire communities of people when there is no work. What happens to the economy when nobody is being paid. And what happens to a country when anything that is paid to corporations gets taken into off-shore accounts?

But, I guess automation has always been a tricky ethical landscape, and ChatGPT is just the latest chapter in that story.

Final thoughts

We've looked at some of the ethical issues and the darker sides of ChatGPT.

I guess my takeaway points are, ChatGPT and other emerging AI's are powerful tools, but they come with ethical and societal questions we can't afford to brush off.

As with any new technology, it's our job to navigating it responsibly.

Be curious, but be cautious.

Thanks, Craig

Post details