My daughter, who has had a degree in computer science for 25 years, posted this observation about ChatGPT on Facebook. It's the best description I've seen:
@DrewKadel She’s right. And that being the case, why on earth is anyone using it? If it’s not providing correct information, WTH is the point?
I just saw a post that cites info from ChatGPT about companies doing business in TN. As if it’s an authoritative source…
@kyozou People use it because it's attractive and talks in language we can understand. Some sophisticated users can get it to do things like aspects of coding-but as a shortcut & they know how to test for reliability. But, good grammar aside, it's not ready for use in the humanities or scholarship. Ironically, its big weakness is that it doesn't give right answers about things found in uncontrolled natural language--though it's all about natural language.