ChatGPT: A Double-edged Sword for Article Writers

Looks like Google thinks it’s ok to use AI for writing articles.


I love my ChatGPT but boy, does it have issues as a writing tool. I’m here today to share some stories from my weekend adventures with my favorite AI and let you know how it wasted precious hours of my time trying to get an article going.

Wait, can we use AI to write articles?

In 2022, John Mueller said that we shouldn’t use AI to create articles. I could see why. After all, most AI generators were based on GPT-3 kinda lame.

Then ChatGPT came along. And looks like Google has changed its tune.

As always, Google says that you need to focus on people, not on ranking on search engines. But the important thing is that you probably don’t have to worry about Google tracking some “AI pattern” in your text.

While I suspect this is partly due to the fact that it’s not easy to identify AI-generated content, I think Google is right in this case. It really doesn’t matter who chained the words together. It’s the value to the reader that matters. There’s no room for “siliconism” when evaluating content.

Back to my story.

ChatGPT, Draw me a sheep

Bonus points if you recognized the sheep reference.

If you’re subscribed to my newsletter, then you already know that these days, we’re pivoting to a different kind of writing. In addition to longtail posts, we’re also trying to create content that will get more traction on social media and possibly on Google Discover. I explained why here.

I decided to try and play around with creating some articles of this new type just to show my team what I mean. I turned to ChatGPT to help me with ideas, outlines, etc.

It’s wintertime. I live in Chicago. So, I thought to myself, why not write something about freezing temperatures?

Here’s what the initial brainstorming looked like at some point –

Cool ideas, right?

But I was looking for something even more interesting, and I came up with my own direction. My silicone buddy liked it!

Now, these ideas, I really liked!

Especially the first one. I figured a story from 1970 would be interesting to revive. I bet no one has written about that in who knows how many years.

Ok, let’s roll with that one!

A Whale of a Tale

I wanted to hear more about that whale story, so asked ChatGPT for an outline, including references. This is what I got. (Mind you, this was within the same conversation, so the AI had context and knew the topic from its own list).

Sounds interesting, doesn’t it? I imagine that back in 1970, the public would have been pretty shocked to see the devastation that plastic was beginning to cause. Cool story! By that point, I didn’t really care that this has little to do with Chicago winters.

Then, I took a break from ChatGPT and went to look for this story on YouTube. I wanted to see if there was any footage from 1970 that was maybe reported somewhere.


Lots of whale bodies with plastic in them (very sad). But no Willy, other than this guy –

Well, maybe there was no video recording of the incident.

And Willy could be a common name for a whale? I think.

I moved on to Google.

Same deal there. No indication of the 1970 frozen carcass of Willy the Whale.

By now, I had spent over an hour working on my article. Frustrated, I returned to ChatGPT and asked, “what gives”?

Well, gee, thank you very much. What a waste of my time.

I decided to try something different.

A Desk-aster of a post

My desk is not very large, so I came up with the idea for a post about the largest desks in the world.

Pretty cool, isn’t it? With links too!

I also have a paid subscription to the OpenAI Playground, so I went there to see if I could get some more desks for my list. And I did!

With links too!

By the way, I don’t know why the playground delivers such different results when they’re both supposed to be using the text-DaVinci-003 engine, but I didn’t really care. I went back to ChatGPT because it’s so much friendlier, and let him know about these new desks that I found.

We then created a comparative table of the desks –

Now, those numbers bothered me. I mean, a desk that’s 300 feet in length? Where exactly can you put such a desk?

I decided to check out those immense pieces of furniture and finally started to click through the links.

None of them worked.

Not a single one.

The sites were there. These are all real websites, but the pages are made up. It’s as if the AI learned what a relevant link looks like and created the kind of links that would look good.

I started Googling for the desks. A few existed but were not that big. Then some desks just seemed totally made up.

As usual, ChatGPT apologized.

By now, I was tired of the desks anyway. Well into the third hour of working with the AI to generate an article, I had nothing.

So, I moved on to a different topic.

The Frozen Felines Fiasco

I went back to frozen animals. I remembered reading about a cat that was found frozen and then revived. Wouldn’t that make for a cool story?

Having lost trust in the AI’s ability to find stories, I decided to try a new strategy.

The Playground version has access to the Internet. Why don’t I research the story first and then provide the AI with links to the sources? All it would have to do is help me write specific sections and save me some time.

I’m happy to report that I found not one but four stories of cats that had been found frozen and then revived. I gave the Playground version the links (they worked!) and asked it to write the story of each of the four cats.

It came back with beautiful stories about these cats, Fluffy, Reginald, Miracle, and Cooper. Lots of drama there, with heartbroken families missing their cats while good-hearted truck drivers rescued them off the frozen roads.

I started putting them into the WordPress post and adding my own input about the dangers of letting your cat wander outside.

Fortunately, I decided to check everything one more time before continuing. I returned to the original stories and compared them to the AI version.

Oy Vey.

Even with access to the original stories, the AI’s mind wandered off like a cat on a stormy night. It even came up with names for two of the cats who were, in fact, nameless strays.

By now, I had wasted four hours of my afternoon.

And still no article!

At that point, I gave up. I grabbed my cat links and put them in a Clickup task, asking one of our writers to pick it up from there.

Does that mean I won’t be using AI for writing?

Heck no. I love ChatGPT. In fact, rest assured it is right here in another tab, helping me craft headings for this post.

It’s pretty awesome for that.

What I have learned from the weekend’s adventures in the land of AI is this:

Do not trust AI with facts. The more specific the story (or niche), the higher the chance for mistakes.

Research your topics. Know them well enough to spot issues. Or hire someone who knows the topic to go over your articles.

It gets more complicated

It’s way too easy to believe the writings of an AI.

ChatGPT writes quite well. And he’s generally a nice guy. He’s the most polite and accommodating writer I’ve ever had the privilege of working with. And he writes FAST!

These things can generate a false sense of trust. And that’s a problem.

What’s more – some web publishers don’t care. I know that many people just use AI on auto-pilot. Whether using a script or getting VAs to prompt it on scale. They pay no attention to fact-checking.

In fact, even if you’re an ethical web publisher, you just can’t tell what your freelance writers might do.

And that’s a huge problem. This would be the low-quality content that Google rightfully does not want to see. Essentially, that is AI spam.

Combatting AI spam

With so many writers, the concern that some might submit low-quality AI junk is real.

Why I don’t think AI detectors are the answer

We’ve tested AI detectors and decided not to use them.

For one thing, I didn’t feel that they were accurate enough. But even if they were – people can easily circumvent them by “mechanically” tweaking the text.

And anyway, Google said they no longer care who generated the content.

What we do instead

Instead of using detectors, we did two things to mitigate the risk of AI spam.

First, we let our writers know that we don’t allow them to submit AI-generated content. We explained why and urged them to thoroughly research their topics without relying on AI tools. We also said they would no longer write for us if we caught them submitting raw AI content.

Secondly, we created a full-time fact-checker role. Our fact-checker goes over successful posts and verifies claims in the posts.

I’m happy to say that so far, so good. No major issues have been found to date.

That isn’t to say that our sites are 100% free of mistakes or even AI-generated errors. But I’m pretty sure it’s mostly good.

But now there’s a new problem

Last week, I came across Google results with misleading content. I’m pretty sure it was AI-generated.

I was looking for information about a ski resort in Illinois. The snippet (also ranking at number one) mentionedit had a ski drop of more than 2,600 feet.

This may sound like a minor slope befitting the prairie state if you’re from Colorado. But actually, Illinois is flatter than that. There are no real mountains in Illinois. A 2,608 ft vertical drop sounded very suspicious to me.

So I checked, and indeed, according to the resort’s official site, they offer a 475-foot vertical drop. 475 and not 2,608.

While I don’t know for a fact that an AI wrote the wrong information, I have a feeling it may be.

This would be the kind of specific information that an AI might get wrong. Not pointing any fingers at ChatGPT here – it may have been a different AI.

The point is: that article was at the top of Google’s search results. And now someone else writing about ski resorts might use that as a source.

We’re looking at the potential flooding of the Internet with bogus information and data. It has begun and will only get worse now that ChatGPT is about to offer an API.

The Internet was always rife with fake news and bad information, but AI could be taking this to a whole new level.

Checking resources is more important than ever

We’ve always trained our writers to choose their sources carefully. We teach them not to rely on other blogs but instead fact-check everything they may read there.

They need to get their facts from authoritative sites such as government sites, universities, official brands’ sites, or the writings of accredited professionals in the field.

This has become more important than ever, so we now ask them to add a list of resources to their Clickup tasks, so an editor can ensure they worked with the right ones.

Is this a perfect system? Far from it. But I feel it sends a clear message and hopefully mitigates the risk of bogus information finding its way into the articles.

It’s not all bad!

As I said, I love ChatGPT. I use it on a daily basis. I have a ChatGPT tab open on my browser at all times.

I use it for –

  • Brainstorming and ideation
  • Tweaking titles and subject lines
  • Improving my writing style
  • Creating general sections in articles

And we have just started teaching our writers to do the same.

At the same time, I think it’s not a good solution for writing whole articles – especially not articles about very specific topics. It can easily trick you into thinking it can do that, but you could end up with an article that’s lacking or just plain wrong.

I realize there are publishers out there who don’t care. They’re happy as long as it ranks on Google – and obviously, it can. IMHO, it’s ethically (and possibly morally) wrong. And it’s a bad long-term strategy for multiple reasons.

Let me know about your experience with AI! Have you found ways to incorporate ChatGPT or other AIs into your workflow? Share your thoughts in a comment below.


  1. Brilliant article, I don’t remember the last time I enjoyed reading such a long article. I agree with pretty much everything. Last week trying to pull some ideas from ChatGPT, 7 out of 10 links to resources it gave me were 404 nonexisting pages. Keep up the great work Anne.

    • Thanks Renzo! I’m glad to hear it’s not just me. I wouldn’t like to worry about a personal vendetta by an AI 😉

  2. Hi Anne,

    I am always so excited when your email comes in!

    I agree with everything you said. Since ChatGPT’s dataset only goes until 2021, I use playground for anything I want better resourced information — I ask it to “cite your sources” in my questions – you’re right about trusting its references 🙂

    But, one thing I’ve learned recently is that to get the most out of ChatGPT (or Playground), we need to become “prompt engineers” (its a just new buzzword circling YouTube or something). Personally, I’m waiting for a better one to get coined and no I haven’t asked ChatGPT to do it 🙂

    So, in my relationship with my new buddy — ChatGPT, my goal is to figure out how to ask it questions (prompting) in a way that gives better results.

    One approach is forming requests with these 4 elements:
    [1] Length
    [2] Topic
    [3] Structure
    [4] Style

    Here is the example I played with,

    “Please write me a 500-word essay on the topic, “The future of politics in Amercia” with a clear introduction, middle, conclusion, along with references and elements of humor written in the style of Mark Twain”

    Please write me a 500-word essay

    a clear introduction, middle, conclusion, along with references

    elements of humor written in the style of Mark Twain

    Anyway, I don’t need to echo your thoughts from this article – but I am excited about the possibilities with OpenAI (et al) and I do think its also about money too. Can we get better quality content produced at $.001 per token or does paying more $.04 per word give us better content or a combination thereof?

    If we choose the former then I wonder if an “all in approach” with human editor review might be the only right change in process. Because, I really don’t think products like GPTZero, Grover, or OpenAI detector can ever possibly do any better than a 50/50 chance of generating a type II error when in fact these algorithms only look at “perplexity” and “length” and do not take into account (a) a writers intent; or (b) a writers knowledge/confidence level on their subject. — Maybe someday, but I am developer too, and I just cannot get my arms around how “length” and “perplexity” can actually do this? I suspect Google knows that too, which is why I also think you are right about how Google is approaching this AI “thing”.

    thanks for your time creating the article and I apologize for the rambling and length!
    Bill Murray

    • Hi Bill,
      I totally agree. The prompts are everything. I will add to this that it’s more than a single prompt. Sometimes you want to work in stages, have CG bring something into the conversation, and then leverage that for context. It’s definitely a skill that needs to be developed. BTW, I did ask my digital buddy and he thinks we should call this “AI Whisperer” I like this! I’ll be using this in my company!
      And yes, I think Google realizes just how complex it’s going to be to accurately figure out what’s AI, what’s not AI and where to draw the line between the two when it’s a hybrid product. We live in fascinating times!

  3. Fascinating times indeed. I think G backtracked on the AI content thing because they’re going to be integrating it with search. I believe we’ll see an increasing number of zero-click search results powered by AI.

    That’s why, like you, I’m changing up my content strategy. I’m focusing on building an audience on social, getting them onto my sites, and ultimately onto an email list. I still publish a lot of content, but am trying to diversify and be less dependent on SEO.

    I think people who are still filling their sites with those “can chameleons eat raspberries” type articles are going to get left out. What the fake gurus call “underserved topics” may end up being served by G’s implementation of AI. I wouldn’t be shocked if Google became more like an answer engine soon.

    • Judging by Bing’s latest escapades, I think we may have some more time before AI takes over longtail, but yes, it will eventually do that. What’s more, competition for these queries is already getting worse, partially due to AI content.

    • Interesting article! I also heard that ChatGPT now has some kind of AI inspector that keeps it from going rogue like Bing has been doing. Maybe 2023 is the year when the revolution started.

  4. OMG this is the best article yet on why it is just not ready to use for content! I love it for topic cluster ideation, title tags (which is a lifesaver for us not so creative types), and meta descriptions! Starting to use it for some Python snippets too – but for content, no way.

    You did a great job of proving that point! 2,608 feet vertical slope for skiing huh? Now that would be interesting!

  5. I believe there is a mistake in your article: “There are no real mountains in Colorado.” I think you meant to say Illinois there. Loved the article btw!

  6. This is a very informative research on the effect of AI on content creation. You really did push the limits with the research and fact checking. Keep it up

  7. Fascinating! I agree, ChatGPT is incredible in getting a birds overview – the outlines and article ideas. The facts are however skewed and need a keen eye to pick out.

    And that is a good thing for niche website owners because the moment Google accepts this is a flaw the sesmic shift in the SERPs will obviously catch many AI content sites off guard. For now, caution is the word when it comes to the use of ChatGPT and other tools in content creation.

    • As far as I can tell, ChatGPT with GPT4 is far more reliable. At least as reliable as the average human writer 😉 Possibly more so. GPT3.5 less so.

Leave a Reply

Your email address will not be published. Required fields are marked *