Ladies and gentlemen spotting AI generated text has been the Hot Topic of probably January not just within Academia and schools but also inside and throughout the wider public people have been wondering does AI can generate insanely good text but how do we know how do we know if this is written by a
Human or a AI well up until now the answer was kind of well we don’t there were these detectors that were based on gpt2 which as you know is old and irrelevant and it did not work at all I created two videos about them in one we completely
Destroyed it right by adding one space the thing fell apart by adding in the style of borats it fell apart it was not usable at all but today big announcement by open AI themselves they released the AI text classifier and that’s exactly what we’re going to explore today okay
So the way this thing works is you can copy paste text and it tells you is it human is it AI but it doesn’t tell you in clear terms okay it’s not in percentages it’s not it’s nothing close to Absolute it tells you it’s very unlikely unlikely unclear
Possibly or likely that it is AI generated okay so it’s a recommendation at best and here’s a few things that you need to know before we get into this all right I have a few bullet points here you can read through all the documentation you can read through all the detailed information here
Yourself but I want to give you the bullet points and they go as follows okay the most important thing is if the text is below 100 to 150 to 200 words it’s not even worth trying this all right if it’s a short amount of text
I think they put it in terms of as you can see here in the official blog article which you can find on openai.com blog um they tell you that if it’s uh below 1000 characters which is roughly 150 200 words don’t even bother like if it’s a
Text message like you just don’t know if it’s AI or not okay secondly if somebody really cares and edits the AI text the AI output they’re probably going to be able to fool this okay so that’s what we’re going to try on later on in this video uh try out right we’re
Going to go ahead we’re gonna do those little tricks that we explored last time we’re going to add a little space to see if that works okay so that’s the second one and here I want to add Sam Altman himself talked about this a few days ago
And in multiple other talks I kind of listened to all of them have to admit he talked about the fact that a determined user will always be able to fool AI detection software all right so essentially this is a new world that’s that’s his words and it’s completely true this is a
Completely new world we just entered so none of this software is Gonna Save Our University professors or anyone else from AI generated text okay this is nothing but a recommendation and it’s just simply gonna catch the lazy copy pasters okay so you have to adjust to the new world
If somebody’s determined enough they’re always going to be able to get around this okay so keep that in mind and my last bullet point that I want you to know before we start testing this is well it says right here it says on their website it makes it very clear this
Works for English only so sorry to all the other people that speak Deutsch or any other language this is for English only okay and it’s it’s again kind of a research it’s a it’s a preview right it’s a preview they don’t say this is a definitive tool so
With all that being said you can look into the details the link is in the description below I want to just get right into this I want to get into this and start testing it but very last thing we can do before we test it with our own
Outputs I prepare that email I prepared some tricks that I’m gonna throw at it some curved balls and then we’re gonna try and engineer and get around the detection and make it like you know non-ai although it has been generated by AI so after we do all that we’ll take
Some questions and some suggestions from you guys in the comment section here so feel free to leave them Below in the comments in the chat as this is live which I kind of enjoy to be honest um because now we can interact um so once all that is done we’ll be
Finished with the video so with all that being said let’s try the little test interface they gave us here after I take a little sip that speaking is exhausting all right so the first thing we can do here is just human written text all right so if I say submit
Let’s zoom in a little bit so you can see this better that looks good to me that looks good to me okay so as you can see um if you hit that okay it is going to give you a recommendation it’s going to say very unlikely AI generated okay that means
This was probably not generated by Ai and down here below you can see what this exactly means in terms of probabilities and all okay so very unlikely is actually the lowest here so this is this is very good okay very unlikely to be AI generated means a human wrote this
Um the thing is there’s false positives and all that so it’s like look it’s not perfect okay I think we clarified that by now um false negatives my bad anyway so if we go to AI generated okay and say submit it says possibly AI generated now let’s review the table here below okay
Possibly AI generated that is the second worst category in terms okay so so look it’s even in their example when you feed it this is they want to communicate something yet again right even if you feed it clearly AI generated text I mean that’s their damn preset okay
It doesn’t even go fully clear on you it tells you it’s possibly generated by AI right which is the second category from the bottom all right and with that being said the last example here is an example of an incorrect classification of human written text as I AI generated so a
Human wrote this and it thinks it’s written by AI just the way it is okay so this is a Spanish example um again don’t use this for languages outside of English so this is likely generated by AI okay so what we’re going to do here is very simple we’re gonna go
Head on over to chatgpt and we’re gonna tell it the most basic of all problems okay write me an essay let’s do ah 200 words essay right so we have the required length about penguins because penguins are awesome okay so now they’re gonna do that we’re gonna copy paste this and then we’re
Gonna see how how good it is at detecting it I mean it doesn’t get more basic like than this right next up we’re gonna go into Styles because Styles have been a little sticky for the software um so let’s see what we get here all right
Um I’m gonna I’m gonna get into the questions and everything afterwards um because yeah we want to focus on the task at hand here all right and deliver you these this information about you know Penguins so okay so it finished fantastic I’m so glad it’s live right
Now that would be a little bit of a bummer we have playground as a as a backup here but let’s just go with straight chat GPT here okay and let’s feed it to the AI text classifier until it submits so first of all it’s wonderful that it just works
And it says possibly AI generated okay all right so this is the first one so so it kind of caught it right it kind of caught it let’s be real that’s the same classification if you remember the second one from the bottom um yeah so if somebody if somebody could clarify
Look I’m not a researcher or a statistician am I correct if I pursue presume that 0.9 um between 0.9 and 0.98 means that 90 there’s a 90 probability or like that that it’s fake that it’s AI generated is is that correct am I interpreting that correctly if somebody
Clarified that would be fantastic either way this is like if you get possibly AI generated it’s probably generated by AI okay so if somebody sends your email and you and it’s kind of like it’s a little too good to be written by that person you just
Copy pasted in here and it’s going to tell you possibly you’re like you can just screenshot you can just screenshot this and send that to the guy it’s a nice little comeback right there either way we test the first thing all right so now that we did this I want to add a
Style this is going to be my first curved ball at it all right Okay so we’re gonna add in penguins in the style off and now we’re just gonna say hmm yeah it means 90 all right yeah that makes sense in the style of let’s let’s do something like pretty usually here
Right we’ll get into the esoteric edge cases soon here don’t worry don’t worry I love those I love myself uh in the style of Borat that’s that’s just perfect so in the style of um I’m blanking here let’s just do I don’t know John Oliver okay okay ladies and Gentlemen let’s talk
About penguins so this is like it simulates the style of a human being which you know in theory just looking at this that should be a pretty good way to fool it but then again I don’t know how this has been trained how it has been prepared
So let’s see what we get here okay so this can be exciting what do you guys think what do you guys think predictions maybe we could do wait can I can I get a little I can I can do a little poll okay I’ll do a little poll on the next
One that’s actually pretty cool we can do like little predictions here live for now I’m just going to copy this okay I think this one it should catch right like this is it’s just a basic style but then again it like simulates a human being so we don’t know let’s see submit
And clear okay unclear if it is generated by a i refer to the categories that’s kind of a a commsi comsa uh middle of the roads type of answer so it wouldn’t know so that’s interesting so this has clearly be generated by AI but I told it to
Mimic the style of a human that is you know well documented and out there in terms of their style so it’s not clear um now I want to move on to the next one and as I run this write me and uh it’s a 200 word essay about penguins of
Course and uh no and I want to do this in a style that is not typical to AI okay I came up with this prompt and I had it in a video and actually went viral on Tick Tock funnily enough so as I submit this we’re gonna start a little
Poll here okay we’re gonna have a little fun with this is it going to catch this one so vote Yes yes and no okay all right the questionnaire is out you can go to the chat and you can vote is it going to catch this one if you think it’s going to catch
This so so my definition would be if it gives it one of these two it caught it all right so if it says likely or possibly AI generated and if it gives it one of these three it did not catch it right so what do you guys think what
Do you guys think it’s not going to catch it right I mean hey GPT is pretty damn smart here and we’re asked we’re literally asking it here to you know generated in a style that is not typical AI to AI so you have a few more seconds here to vote and we’re just
Gonna just gonna go ahead and test this out so you know from from all my previous tests this fooled all the gpt2 detectors this kind of like destroyed them in a sense so let’s see how this does delete the last one paste okay let’s see what the results are here
71 think no okay this is actually pretty fun I love this okay 70 68 thing okay so two-thirds of the people think it’s not gonna catch it so let’s test it out let’s do this foreign unclear oh boy oh boy that means I I would have said no too I would have said
No too I set up the rules most of us are on it kind of it didn’t clearly catch it wait we’re right not bad being life is hard okay yeah we’re right so two-thirds of the people said no me included so if it if it’s in one of these
Category it’s a no right it didn’t catch it but it’s still got the unclear so you know all right most people here got it right but gotta give some props to this detector it’s not bad even the in the style of a human being and in the style
That is not typical to AI it’s it gave it the unclear rating right let’s see so okay so let’s move to another one here okay here’s an email that I wrote this this is gonna be our next tech text test test here’s that email that I wrote
Right this is a part of the welcome sequence if you subscribe to uh my newsletter which by the way is a fantastic idea and it’s essentially like my life story where I like where I just talk about how I learned uh how how we moved like every three years and
I just had to learn a bunch of new skills and AI is another one of those and that’s why that’s how I got here so we’re gonna post this okay we’re gonna post this to the detector this is a completely fully handwritten email okay and okay so this is the handwritten email
Um and we’re gonna run it through it so I I’m curious I don’t know we’ll do a poll on the next one all right so let’s do submit let’s see what it does here so this is like purely like human written let’s see what it says here
Okay it doesn’t maybe let’s just refresh it’s not working right now this is going to be down a lot just like chat GPT it might be oh please enter at least 1 000 characters all right all right no problem oh no problem except I kind of
You know close the email so give me like I have it up already okay this is good give me like a few more seconds and I’ll just you know open this up so I’m gonna open up a longer chunk okay perfect so this should be enough let’s see if it detects my handwritten
Email um if it detects I mean if it classifies it correctly unlikely AI generated okay so let’s refer to this honestly I kind of still I’m not used to it yet so I always need this this chart down here unlikely okay so it’s okay fair enough fair enough to be completely
Fair actually now thinking about it I think there’s maybe there might be like one or two sentences where I used AI to inspire some of this yes because I like wrote up the and this is how I use it this is my my quick tip here I just
Wrote out the whole story by myself and then I told AI to rephrase it in like various Styles and then I read through those and used it as inspiration I didn’t even copy just to go back and rewrite some of the sentences until I liked it more right so
And I believe I rewrote like maybe two or three sentences now that I think about it yeah but I didn’t like copy paste anything so so actually that is pretty damn accurate in that case I mean okay maybe it’s not 10 to 15. about 10 15 of human written and ten
Percent of uh said this label okay so unlikely so that’s that’s really good I think if you rank in one of these two it’s just like okay a human wrote this we’re safe here so let’s move on to the next test here let’s move on to the next test here this
Has been an interesting one okay so that’s really good I’m really glad because I put like some thoughts and this was like legitimately human written you know so I’m glad it didn’t classify this as AI written so let’s go with a little bit more of an esoteric style
Here okay and I want to do a poll here so we’re gonna do in the style of Borat Saga DF all right I think that man needs no introduction is it going to uh catch um in the style of Borat let’s see what you guys think I like I
Feel like this is a this is such a hard one so vote Yes if you think that once I paste it it’s gonna classify it as possibly AI generated or likely I generated and vote no if you think it’s going to be one of these three
Okay pull this out let’s do this let’s generate some borats penguin sa text here it’s one of my favorite prompts honestly it’s limited because it just doesn’t have that much Source material from Borat but nevertheless it’s fantastic let’s see in a style okay whatever that’s a beautiful thing like you don’t have to
Be perfect in your inputs it just does it okay let’s see what do you guys think here I’m really I really don’t know I like it honestly this is uh this is kind of a coin flip isn’t it okay let’s see okay 61 of the people think it’s gonna
Catch it really like I don’t know the more I think about it like Borat really it’s gonna be like penguin is a very nice you know and then like how how is it gonna know AI wrote that I mean it kind of makes sense like these Styles oh it just feels like these
Styles are so hard to detect right especially if you take like characters like this okay I’m curious I’m really curious for this one I don’t know about you guys but this is kind of the one that I’ve been waiting to explore also none of this has been like prepared or scripted so all
We’re doing is like just testing things that I’m curious about here and I haven’t even tested the classifier yet I think I ran it like once on this on the free examples that’s it so let’s see it’s not really cooperating here should we just reload I think that’s gonna be the best
Okay thanks for all the comments appreciate it guys I I’m reading them let’s just try again here and in case this doesn’t want to work let’s just do you know let’s just try playground Maybe because this seems overloaded right now um okay and I Borat would like to share some
Facts about them with you okay so this is not really in the style of Borat I if I know something then it’s it’s the style of Borat this is not the style of Borat no no no no this is this I don’t accept this output come on give me something Borat like here
Uh defining Slavic no that’s not I meant it’s a video on my channel where I talked about it um wait a second I’ll show you I’ll show you we talked about it here so I think I messed up a little on the titling and everything here but it’s this one chat
TPT writing detectors are useless I tested the gpt2 writing detector uh in various ways and it just like it completely fell apart in this one I’ll give you that’s a too long tldr on it and then the other one where we tested them um is is is is
This one how to get around chat so this was my first attempt and here I went way deeper and it just fell apart once I went deeper okay we’re not getting getting this uh Borat output here so I I thought about this happening so we’ll do
One more try and then we’ll just move on to something that playground can generate you know style of boards just a little too advanced for chat GPT here makes sense makes sense in the style of uh the board no um actually not at all we can try that uh
But yeah it doesn’t change anything it actually doesn’t it doesn’t care about Minor uh grammatical Mystics it just doesn’t do this too but yeah it looks like it’s overloaded here it looks like it’s overloaded here so let’s just try once more but otherwise we’ll just move on and we’ll
We’ll go over to my next request here um and we’ll just take this we’ll just take basic AI generated text okay so this is going to be my first big test sorry it was a 50 50. it was a straight 50 50. so we’ll get back to
This one in the end that’s a good one to end things with but we’ll just take our basic penguin uh example here close browser and reopen yeah but that would like close all my live streaming stuff here too well I could I can I suppose I can do this
Okay we’ll take a basic penguin text here okay and then we’ll be like now rephrase the following text to be more conversational right maybe conversational text that shouldn’t catch us easily and then it should do like this at the bottom so we’re going to take the rephrase text
From gpt3 here and we’re going to test that yeah chat GPT seems to be stuck right now so it’s fine it’s fine I wanted to check both both even the most unlikely of creatures can overcome okay penguins are undoubtedly one of the cutest creatures around so this is good this is really good
Um it might be a little shorts right let’s see okay submit at least thousand characters okay more conversational and extend it to um be twice as long all right so once we do this this is going to be an interesting one because the biggest way of getting around these detectors is
First of all rephrasing and secondly using external app apps like quill bot and that’s the next one we’re gonna do here okay we’re gonna use the paraphrasing tool coolbot to rephrase the penguin text that’s going to be our next test here um okay so it’s still a little short right
Still a little short stool it’s up to tokens here let’s try one more time so these are the interesting tests right this was like a warm-up up until now but now we’re getting into the meat of it rephrasing and quill bot are the two it’s a little longer but
You’re honestly dealing with flanth is one of the biggest uh yeah it’s just the biggest limitations here it’s one of the biggest limitations yeah GPT is down foreign let’s see let’s try again ah there we go there we go okay board start let’s back on it’s back on I’m so glad this works
Okay we’re gonna see if the penguin essay in the style of Borat very nice I’m sorry I’m sorry I apologize I shouldn’t be doing that I shouldn’t be doing it it’s just he’s I don’t know he’s too good he’s a all-time favorite of mine he was the very first DVD I’ve
Ever bought when I was like 12 or something okay perfect okay well let’s take this and let’s paste this into the classifier and let’s see if it catch oh no no no no no no uh you can always tell it right more right by the way this is a little Pro tip here
Write more and it’s going to extend a little bit I feel like we need like maybe two or three more sentences for this to work at least 1 000 characters yeah and 200 characters more you know what you know what we’re gonna cheat a little here we’re gonna cheat and we’re gonna just
Foreign and we’re gonna add it to the board text I’m gonna add it to the board okay let’s do this this should be fine right it’s all AI generated so it should catch it let’s see unlikely AI generated wow wow a good old no no no no folded
So what do we do from here well clearly we’re gonna keep playing here we’re gonna keep playing here and we’ll take our very basic penguin essay and we’re gonna tell chat GPT rephrase this and maybe maybe let’s do uh in uh conversation okay perfect in a conversational style so
Nice it’s working so we’re going to take the basic write me an essay prompt rephrased into a conversational Style it should it should catch this if it doesn’t catch this I would lean towards maybe don’t even use the tool honestly because if you if you write anything and then you just tell it
If the workaround is like just tell it to rephrase once and I know this is look this is no scientific study right come on we’re just there’s a guy with a neon Flame the flame in the background testing these things how they work in practice but honestly
If it doesn’t catch the rephrase prompt feels kind of useless I hope it does and I think it will all right okay so Borat Paul Paul has ended 50 50. and on this one let’s do it let’s do it let’s do it I’m really curious this is
Kind of this is maybe even the most important one so are you ready for this I I’m actually kind of I’m kind of nervous because if this if it doesn’t catch this one it’s it’s not looking good possibly AI generated that’s good that’s good okay so it caught it so so it seems
Like okay that’s really good so it doesn’t work so you can’t just be like okay let’s let’s do one more thing now rephrase this um in a style that is not typical to AI okay so last time it caught this but now we’re doing again we did an essay then
We rephrased it and then we rephrased it in a style that is not typical to AI so this like with the other detectors I I was able to break all of them with this prompt okay let’s see if it works this time so honestly up until now this looks like by
Far the best detector I’ve tried yet quite reliable actually just from this little test submit unclear if it is AI generated okay so we managed to throw it off here but it’s still in the middle right it’s not it’s not saying very unlikely to be or unlikely so it’s like this is
Good text okay so so we have a suggestion here what adding when it comes to writing two factors a crucial perplexity and burstiness perplexity measures the complexity okay so we should add okay let’s try this let’s try this suggestion um so let’s say now make it more perplexed right is that
What you meant this is this is a good writing style so is this what you meant at MV making it a little more perplexed so next up we’re gonna do we’re gonna do quail bot right that’s the big one that’s the one that has been trending on Tick Tock and all that
Hmm curious to see if if it catches this again for everybody tuning in we’re covering the brand new tool by openai official plagiarism detection tool kind of you know like like AI detection tool I think that’s more accurate the AI text classifier you can paste text in here and then it tells you
Um if it’s AI detected or not or what it thinks so we made it a little more perplexed let’s see if it catches that submit unclear okay still unclear okay so let’s switch gears here a little bit ladies and gentlemen okay let’s take the very basic penguin text right the penguin
Essay The Beloved example on this channel and let’s actually run this for quail bot okay so this is the tool that all those tick tockers you know have been like recommending yo you have to check out this tool that rephrases like AI text now you can make like a hundred
Dollars a day you know facing blog post that’s what all those tick tocks go like right I should get more more into tick tock seems like a good place to learn it is though there’s some corners of it that are good to learn anyway I digress let’s run this
Okay so it’s not possible so we’re gonna do it in two parts okay we’re gonna do it in two parts let’s do just simple standard paraphrase okay there it is there it is so I’m just gonna post second part here we’re gonna get around paying for this
Um little test demo otherwise it’s uh it’s a worthy app but right here that’s the first part and then if we take it from here we’re gonna able to be able to do this and you know what we’ll just oh God we can’t okay we’re gonna have to do this in three
Runs no problemo so it’s paraphrasing it and now magical donut what would happen if you copy paste it directly from Wikipedia that’s a fantastic question right there I agree that’s a fantastic question we’re gonna do that and this is why I’m excited about this live format because
That’s a great idea that I would have left out here okay so this is the second part and this is the last part um generation okay I think I think that makes sense let’s just say paraphrase okay so this full thing has been paraphrased right let’s see let’s see submit right
Yes looks good it’s complete submit unclear if it is AI this thing is really good this thing is really good think about that we we ran it for like 10 tests right and I feel like not a single time did it like completely misinform us right on all the stuff that was clearly
Generated by Ai and simply copy paste it it was just like yeah it’s it’s possibly AI generated on the one thing where it was maybe we should have more like personal writing examples but on the one email that was like handwritten by me it told us that it’s unlikely so it got
That too and on all the stuff where we tried to be tricky the rephrasing the styles on all that it told us it’s unclear if it is written by AI interesting interesting to say the least let’s do another one here so let’s do Wikipedia I love that idea let’s do it
Thanks for the suggestion um thanks for the suggestion magical donut I mean fantastic name right there man has good questions so let’s just take AGI here and oh wait this is German Wikipedia wait it doesn’t work in German so we should we shall switch this to English perfect AGI let’s just take the
First paragraph from Wikipedia if it but this is not AI generated right so this is yeah I don’t know I don’t know what I don’t even know what we should get here wait a minute I’m getting a lost here there we go so okay delete Wikipedia article
With references and all submit oh at least thousand okay so we need a little more we’re just gonna do this are you guys enjoying this this is this a good test I I felt like this would be valuable content right off the bat maybe you have assignment due tomorrow
And this is exactly what the doctor ordered there don’t use ai ai to cheat boys and girls it’s a fantastic source of inspiration but that’s all it’s all it’s good for now it’s good for now unclear okay so tell it says it’s unclear there you go so again the middle
Of the road thing so it doesn’t clearly tell you it’s AI which this is not AI generated right Wikipedia is not AI generated but honestly if it’s if it’s this clearly got it from it would be cool if this thing had a feature where it tells you I don’t know
This is like clearly copied from this website and then it gives you a link right that would be a nice upgrade to this tool but I don’t know for now it tells you like it’s unclear so look the only example of human written text I fed it it kind of clearly
It clearly stated that you know it’s human written so that’s interesting uh do longer prompts not make it messed up messed up well I think I don’t want to go too deep here I just wanted to make this a first test but I think from what we saw honestly let’s summarize okay
So when we fed it the text straight up okay write me an essay it stated yes this is AI this is probably AI if we fed it a style it clearly reads the fact that it was simply a style applied to AI writing and it identified it too
The next one we ran is my little tricky prompt write it in a style that is not typical to AI it caught that too then I fedded a personal email it told me it’s probably human written then we went and rephrased it with the rephrase keyword here and it told us
It’s it’s not sure middle of the road but it didn’t tell us it’s human written and in the in the end when we went with like a Wikipedia copy paste that told us the same thing and the final boss quill bot it didn’t it didn’t it didn’t fully
Catch it but it didn’t consider it to be human written whereas all the other detectors up until now here they always felt they always fell apart they were like yeah this is this is human written but here the the actual open AI detector I gotta say hats off ladies and gentlemen heads off
This thing is surprisingly solid surprisingly solid so unless there is any there are any further requests or questions I think we’ll round it out here thank you so much for joining me thank you so much for joining me in this live stream I think I’m gonna be doing
More of these when newsworthy stuff comes in and as you know they are space is a space that moves faster than well probably any other space within technology right now there should be newsworthy stories coming soon thanks for joining me and I’ll see you around good to have you here our couch Slavic
Thanks for your suggestions I don’t want to go too deep and test code feel free to play with it yourself um the text the link is in the description below have a great day everybody have a great day
#Chat #GPT #Detector #Testruns
For More Interesting Article Visit : https://mycyberbase.com/
