Unpublished study indicates high chance of fraud in thousands of tests of enem

4/23/2018 Estudo inédito indica alta chance de fraude em mil provas do Enem - 23/04/2018 - Educação - Folha
https://www1.folha.uol.com.br/educacao/2018/04/estudo-inedito-indica-alta-chance-de-fraude-em-mil-provas-do-enem.shtml 1/13
Unpublished study indicates high chance of fraud in
thousand tests of Enem
Folha's research analyzed 3 million templates, from the 2011 to 2016 editions
23.Apr.2020 to 2:00 p.m.
PRINT EDITION (//www1.folha.com.br/fsp/fac-simile/2018/04/23/)
SÃO PAULO, RECIFE and PICOS (PI) Unprecedented statistical analysis points to a high
probability of fraud, in diﬀerent forms, in at least 1,125 tests of the Enem
(http://search.folha.uol.com.br/search?
q=enem&site=online&results_count=4556&search_time=0.054&url=http%3A%2F%2Fsearch.folha.uol.com.br%2Fsearch%3Fq
%3Denem%26site%3Donline&sr=126) , a national exam that selects students for public
universities in the country.
These tests are within groups with a pattern of responses so similar to each
other that, statistically, it is unlikely that there was some kind of glue in these
cases.
According to the statistical model developed by Folha , the chance of these
tests being similar only due to chance in an edition of Enem is at least 1 in
1,000.
ADVERTISING

That is, it would be necessary to repeat the test a thousand times so that two
tests, without interference, were as similar as the suspect jigs.
Joint investigations by Inep (federal agency responsible for Enem) and the
Federal Police have so far confirmed only 14 cases of fraud. Management
Michel Temer says he uses statistics and other means to fight queues.
Folha's study identified both double suspects, which indicates some type of
rudimentary glue, and groups with up to 67 suspect candidates, pointing to a
more sophisticated scheme of transmission of responses.
The survey considers only candidates who were among the top 10% of the
marks, between the 2011 and 2016 editions, which represents a total amount
of 3 million tests analyzed. With this score, the candidate can enter into
courses such as medicine, law or administration.
The model adopted is more rigid than that applied in other studies that
sought to identify fraud in exams and public competitions.
The statistic was used, for example, to detect glue at a US Air Force university
last year, or fraud in a contest for a seat at the Federal Revenue Service.
The Enem charges 180 candidate questions, with five alternatives each. The
Folha survey calculated the likelihood of two or more tests having the same
pattern of correctness and error.
It was considered highly suspicious, for example, candidates who missed
questions marking the same wrong alternative (they could have mistakenly
choosing other three equally incorrect options).

Another point considered was the likelihood that these high-income students
will err on the issues of medium or easy difficulty. If they fail to answer these
questions, it becomes more unlikely that the similarities between the answers
will be haphazard, as they are expected to answer these questions.
Five foreign language questions were discarded because students may have
different patterns of responses (they choose between English and Spanish).
The information processed by Folha is official, called Enem microdata. The
bank shows all the answers of all the candidates in the exam, where they took
the test, where they resided at the time of the evaluation and other data.
For the past three months, the report has sought unexpected patterns in
those responses.
The database does not show name of the candidates or any identification
number like CPF or RG.
Crossing Enem's information with those of university entrants, the report was
able to identify a candidate whose evidence is among the suspects.
Justimar Leal Teixeira, 48, had in the 2015 edition of responses very similar to
that of 24 tests.
To find a set of tests so similar to chance, it would be necessary to apply one
edition of Enem per year since the creation of the universe (14 billion years
ago).
For ten of these suspicious tests, his feedback differs in only 5 out of 175
questions analyzed, including more than 30 wrong answers in the same
alternative.
If the similarity were only a random one, the model indicates that there
should be at least 39 divergent questions.

Teixeira was born in Patos (PI), a city of 6,000 inhabitants. Two other
suspicious tests were done by natural persons from the same city.
Asked about the similarity of his feedback with the others, Teixeira stated that
"if you get the writing, there are differences" - the report had questioned the
objective part, not the writing.
Teixeira denied that he participated in the scheme. "In a large universe, there
can exist [similarities]. How many people do Enem? "
The statistical model adopted by Folha already considers that the
examination is done by millions of students, which increases the likelihood of
two or more similar tests, even if there is no fraud.
But the judgments considered suspect are so similar that it is unlikely that it is
merely the work of chance.
Teixeira was approved in 2016 in biomedicine at the Federal University of
Pernambuco, in Recife, but has the course locked, after attending two
semesters. He says the job, as a lawyer and a military man, does not allow him
to go on to graduate.
The statistical survey also identified a student with suspected evidence, within
the same group of Teixeira, who entered medicine at the Federal University of
the Valley of São Francisco, Bahia, in the second half of 2016. The report could
not contact her.
3 million: Number of tests analyzed in the statistical study
1.125: Number of evidence identified as suspicious because they are very
similar
1 in 35,000: It is the average probability of finding such similar evidence at
random (ie, Enem must be applied 35,000 times for the result to be repeated
without interference)

254: Number of cities in which at least one suspected test was found between
2011 and 2016
26: Number of states where there was at least one suspicious test (only
Roraima did not appear)
21: Schemes found with more than three tests (which suggests a more
organized system than one student gluing from another)
The group with the highest number of suspected candidates was identiﬁed in
the 2016 edition of the exam, with 67 tests spread across ten states
(Northeast, Southeast and South).
Picos (PI), with 77 thousand inhabitants, was the city that most concentrated
cases of this group, with 11 tests. Eight of these candidates traveled to the
municipality to take the exam, four of them from Teresina (a route that takes
more than four hours by car).
The Enem is applied in more than one thousand municipalities of the country,
including Teresina and other surrounding cities.
Two evidence from people who traveled from Teresina to Peaks had 115 equal
questions, 48 wrong in the same alternative and only 12 divergent.
Randomized simulation, with performance similar to this group, indicates
that there should be at least 62 divergent.
The two suspected exams were done by men aged 34 and 29 at the time. We
were unable to identify their names.
Regional Delegate of the Civil Police in Picos, Jonatas Brazil said that it did not
receive news involving fraud in the Enem.
Over the years, the Federal Police and the Federal Public Prosecutor's Oﬀice
have started operations that identify gangs
(http://www1.folha.uol.com.br/educacao/2017/11/1933736-policia-federal-faz-operacao-em-tres-estados-contra-fraudes-no-

enem.shtml) that act to pass certain answers to candidates who hire the service,
both in the Enem and in public tenders.
The investigations indicate that the contractors in general have interest in
entering medicine in the public universities. They even pay R $ 180,000 to
receive the right answers.
The schemes discovered(http://www1.folha.uol.com.br/educacao/2017/11/1934910-operacoes-da-pf-contra-
fraude-no-enem-tem-59-buscas-e-5-prisoes.shtml) had "pilots", usually university students or
teachers, who quickly solved the test, trying to get as many questions as
possible (it is almost impossible to hit the 180 questions).
They then forwarded the responses to the other candidates, via radio or
cellular messages. Customers used point or cell phone, although prohibited.
The transmission was complex. In Enem there are four types of evidence,
identified by color. The order of the questions changes on each test color.
If a driver solved the yellow test, he could not simply pass the customer
feedback with the blue test.
One of the possibilities used by the fraudsters was to speak on the radio the
first word of each of the 180 questions and the word that indicates the right
alternative. It was then up to the contractor to find these options in their trial
version.
Complexity makes it difficult for proofs within schemas to be exactly the
same.
The Federal Police found a gang like this in the 2016 edition of the exam. The
answers were passed by pilots who were in Montes Claros (MG) for candidates
in Minas, Bahia and Ceará.
Another two schemes were issued by the police at the end of last year, when
42 enforcement orders and 5 arrest warrants were issued (one group

investigated by the PF team from Juazeiro do Norte in Ceará and another by
the Pernambuco police team) .
Folha's study shows that there is a high chance that identified groups may
have acted in other years as well.
The PF identified suspects in the cities of Juazeiro do Norte, Brejo Santo and
Barbalha in 2016. The statistical model pointed to suspicious evidence in the
same cities in 2016, 2015 and 2013.
The statistic has been used in Brazil and abroad as a tool to detect fraud in
exams.
In 2017, the Justice confirmed in the second instance suspension of candidates
for public tender for financial analyst of the Internal Revenue Service.
The report pointed out as statistically unlikely that it was only coincidence
the similarity in the test of 28 candidates. In 170 questions, some had the
same 122 correct and missed 40 marking the same alternative.
The applicants, who have appealed since 2005, say that the statistical model is
not enough to determine that there has been fraud. So far, the argument has
not been accepted.
In the United States, one of the most emblematic cases involving the use of
statistical models in examinations occurred in 2009 in Georgia (USA).
Research from the Atlanta Journal-Constitution newspaper pointed out that it
was statistically unlikely to vary grades of some schools in the test applied to
state students as they rose and fell dramatically in a year.
Later government report pointed out that 38 directors and 140 teachers had
changed students' responses to improve grades.
Last year, suspected evidence fraud was analyzed at the University of the
United States Air Force.

Statistical analysis showed that two people had the same answers in 289 of
300 questions, distributed in six exams. The case was analyzed by a
committee that monitors proceedings against federal servers.
"It was more likely that the US lottery winner would win the next four raﬀles
than these results would happen without interference," said Dennis Maynes,
chief scientist at Caveon, which serves institutions such as the Florida
Department of Education, IBM, and Microsoft.
Maynes did a study that based the trial and created a model that inspired
Folha's methodology . "The statistical evidence was enough to convince the
judges that there was cheating," said the American company scientist who is
looking for fraud.
He states that, in other cases, statistics can be seen as the initial step for a
broader investigation.
The report consulted the scientist for a veriﬁcation of the statistical method
adopted to analyze the Enem.
Maynes suggested that some criteria might be less rigid, such as the threshold
for considering evidence suspect.
The report took the threshold of at least 1 chance in 1,000 to consider the
candidates as suspects.
Maynes said it could be from 1 chance in 100 (used in some academic
research).
In this format, the number of suspected exams in the Enem would rise from
1,125 to 1,513. The sheet , however, preferred to emphasize the most
conservative model.
SOME FRAUDS DISCOVERED BY PF

jan.2018
In Enem 2017, a 27 - year-old student entered the test room with the hidden
cell at the waist and copied the excerpt from a book in writing
Where: Bahia
Nov.2017
In evidence from previous years, the gang passed from the outside, the jigs to
the candidates. In the operation called the Easy Pass, 31 search and seizure
warrants and 31 coercive driving orders were carried out
where: 13 states (PE, BA, ES, GO, MA, MG, MT, PA, PI, PR, RS and SP )
Nov.2017
Quadrilha Fraudava , besides the Enem, other competitions; 28 search and
seizure orders and 11 coercive driving orders were served, with ﬁve prisoners
Where: Ceará, Paraíba and Piauí
Suspected evidence of fraud in Salvador - Divulgação / Federal Police

Sep.2017
Thirteen people were indicted for diﬀerent types of fraud in the 2015 and 2016
tests; there were cases of candidates who knew the subject of writing
beforehand, took a written glue or even pretended to be Sabbatarians
(religious who need to keep the Sabbath)
Where: Maranhão, Pará, Amapá, Ceará and Piauí
Indicted arrested in ﬂagrant in Amapá for fraud - Disclosure / Federal Police

Nov.2016
Quadrilha charged up to $ 180,000 to pass answers through an electronic
point connected to a chip card, attached to the body, that functioned as a cell
phone; the candidate conﬁrmed if he understood with a cough
Where: Montes Claros (MG)
Man carries electronic points on body to fraud Enem - Play / TV Globo

Nov.2014
Group also used card scheme to pass feedback, from a hotel, through pre-
arranged codes. There was a handbook that taught how the candidate should
act. An agenda indicated more than 160 students who would have been
approved
Where: Pontes and Lacerda (MT)
INEP SAYS THAT IT WORKS TO COVER FRAUDS AND GUARANTEE ISONOMY
Arm of the Ministry of Education responsible for the Enem, Inep said that he
works with the Federal Police to curb fraud, ensuring "isonomy among
participants."
The institute said it also makes statistical analyzes aimed at combating fraud.
When there are suspicions, the cases are referred to the PF for further
investigations, because "the announcement of the Enem does not foresee the
elimination of the participant [only] by coincident results."
Suspects held over fraud in Enem - Playback / TV Globo

The official note did not say how many cases the police had reported to the
institute.
Inep only stated that the cases of fraud already completely confirmed by the
Federal Police "are punctual": 14 in total, being a candidate in 2013 identified
in Minas Gerais, and 13 identified by the team in Maranhão (3 cases in 2015
and 10 in 2016 ).
"These cases did not compromise the smoothness of the examination, from
the assertion of the delegates responsible for conducting the inquiries that
the exclusion of participants would be sufficient provision, which has already
been adopted by Inep," the institute said.
The agency says it has upgraded security systems in the test, such as the
adoption in 2016 of metal detectors at test sites and biometric identification.
And, last year, with electronic point detectors.
"There are other police investigations going on. If there is an indication of
fraud duly formalized by the police authority, the participants involved will be
eliminated from the Enem, without prejudice to other measures, "the body
added.

Unpublished study indicates high chance of fraud in thousands of tests of enem

Recommended

Recommended

More Related Content

Similar to Unpublished study indicates high chance of fraud in thousands of tests of enem

Similar to Unpublished study indicates high chance of fraud in thousands of tests of enem (19)

More from Caveon Test Security

More from Caveon Test Security (20)

Recently uploaded

Recently uploaded (20)

Unpublished study indicates high chance of fraud in thousands of tests of enem