ChatGPT3.5 VS ChatGPT 4.0 – Same prompt VERY different results
ChatGPT is very often very wrong. I learned this the hard way.
However, in my experience it is getting better. The goal was to have ChatGPT write a PHP script for a little project.
The script in question was supposed to trigger an email campaign in MailChimp every morning at the same time. I’ve confirmed multiple times that MC cannot do this out-of-the box.
Seems like a straightforward script. Here’s my first prompt:
…and here’s ChatGPT’s response.
Ok, at first glance this looks legit. I always like how the reply often gives some extra information like making sure cURL is enabled.
However, there are some issues.
Issue #1: This is technically impossible. You can ONLY trigger a campaign if it’s status is “Draft”.
Issue #2: Once it is triggered, it cannot be triggered again.
Issue #3: This will only send to one subscriber at a time, as noted by the inclusion of the subscriber hash.
In other words, this could work for a draft campaign if the goal is to trigger the send one time.
It would also work if you were programmatically creating a campaign and then using this script to trigger the send – so the code isn’t actually bad per se, it’s just not exactly applicable for a daily newsletter to the entire audience.
I wasted a good bit of time digging into this since ChatGPT was so confidently incorrect and leading me to believe it would work.
Here’s what’s interesting about this outcom:. Scroll back up to the original prompt… the header says: Model Default (GPT3.5). Let see how Model GPT4.0 will do with the same prompt.
Eureka! GPT4.0 instantly pointed me in a different direction, stating that what I was trying to do was not really the intention of the API and how to do it differently.
Now, this wouldn’t work for my exact purpose since I’m using the MailChimp feature “Import HTML from URL”, which is only available inside of a campaign and not in canned templates (for some reason). This was confirmed with MailChimp support too.
Back to the drawing board to get this automation set up properly.
Biggest lessons so far – be skeptical with AI output. It may seem accurate but might be outdated or misrepresenting whatever code (or content) it generates.
Also, instead of banging on a stubborn platform, try finding an alternative. Sometimes it makes sense to hack whatever you have but observe when to try a different tactic all together.
Stay tuned for the next part: Hacking MailChimp to send a daily email!