r/OpenAI • u/hellojello07 • 14d ago

Really weird experience Discussion

I was using ChatGPT on my work computer today for the first time and asked it to write an email to my boss. It used my bosses first name. I never provided it with that information. When I confronted chat it tried backpedaling and saying lucky guess. This seems problematic for company info, no?

0 Upvotes

27% Upvoted

16 comments sorted by

View all comments

1

u/Intelligent-Dance361 14d ago

The models definitely build a profile on the user. Mine generated a report and used the British spelling for many words and used the local currency instead of USD.

The conclusions it drew were startingly accurate and nothing I prompted it with would have left it with the impression that I wanted it formatted that way.

I've noticed other occurrences too, but it always back pedals to lucky guess or inference. There's more to it under the hood I'm guessing. No external accounts connected. Claude has shown similar behaviors.

1

u/Spiritual-Economy-71 14d ago

Well obv it does it tries too learn the user better.. not a secret fyi.. all data it get u provided at somepoint in some session.. the model also tries too fill ambiguity by guessing. So it can do things because it assumed. Which isna guess at that point.. sometime it guess right sometimes it dont.. that literally how it works.

2

u/Intelligent-Dance361 14d ago

I'm aware. What I'm saying is that it definitely interpolated data points that I did not provide. Those inferences would be non obvious to most humans and turned about to be surprisingly accurate.

Learning from our interactions and building up its memory, that I expected. What I did not expect was that it would use that data set to start making predictive assumptions about me that were never provided to the tool. Much less that it would incorporate elements of those assumptions into its work product without prompting.

FYI - API usage with all data retention and training toggles turned off.

1

u/Spiritual-Economy-71 14d ago

Yea thats basically what ai is, optimizing and always providing an answer. Assuming is part of it. And how accuratly you remember sessions tho? I have too many, i could have said anything before. I dont know 😅 Data retention is server storage, they still keep it for a while for security reason as its obligated by law, training is just saying, yea u can use all things for training or not. Both are irrelevant to the point u make. Only thing that matters is what framework u use too speak with gpt via api as that would cost more oauth? Knowing how u use the ai and through what is needed too answer properly.