In a research context, artificially increasing image resolution through interpolation of additional pixels would generally not be considered appropriate, as it introduces artificial data that was not present in the original image. Minor sharpening could potentially be acceptable to improve clarity, but adding whole new pixels risks misleading viewers about what is actually depicted in the original data. The best approach is to capture images at a sufficiently high resolution during data collection.

1. Editing Digital Imagery
in Research:
Exploring the Fidelity-to-Artificiality Continuum
Dr. Shalin Hai-Jew
Kansas State University
CHECK 2021
May 20, 2021

2. 2

3. Various Junctures at Which Errors May be
Introduced (Awares or Unawares) (and Magnified)
3
• Project Setup: literature review, research design, team seating, work
delegation (and crediting), research oversight, representations to
funding agencies
• Project Execution: research, fieldwork; data capture; data recording;
data archival; data cleaning; data storage; data analysis; data
representations; technologies; resources
• Reportage: conference presentations; publications; data sharing
• Post-Release Vetting: double blind peer review; data review; follow-
on studies; administrative review

4. Common Risks and Challenges to Research
Integrity
• in a context of…
• career (non)survival;
• time/budget/equipment limits;
• limited tools and limited resources;
• difficult and complex work in a complex environment;
• competing colleagues who seem to be doing better;
• competition and mutual advantage-taking;
• impression management, etc.
4

5. Common Risks and Challenges to Research
Integrity (cont.)
• Dishonesty, Over-Claiming,
Misrepresentations, Exaggerations
• Inappropriate Delegations and
Handoffs (~ ghostwriting; data
analytics as “scut work”;
commercial pre-written papers)
• Poor Work / Unskilled Work /
Rushed Work / Corner Cutting /
Carelessness / Incomplete Work
• Non-Expertise / Insufficient Skill
• Incorrect Data Cleaning and / or
Data Removal
• Conflicts of Interest: Nepotism,
Bribe-Taking
• Staging, Re-enactment, Enactment
(in In Vitro and In Vivo Research)
• P-hacking / venue shopping
• Rejecting Unexpected Research
Results
• Sabotage (acts of malice)
• Data Corruption / Data Alteration
5

6. Common Risks and Challenges to Research
Integrity (cont.)
• Data Fabrication
• Plagiarism / Derived Works /
Lack of Originality (and Non
Crediting of Others)
• Credit-Usurpation Free Riding
• Data Leakage or Mishandling
(confidentiality, PII, anonymity,
NDAs, and others)
• Premature Release of Research
• Misappropriation of others’
ideas and works, published or
not, including from privileged
communications
• Funds Misuse
• Real-World Contingencies and
Accidents and Losses (and
Mitigations or Non-Mitigations)
6

7. Common Risks and Challenges to Research
Integrity (cont.)
• Poor Data Stewardship
(technological obsolescence; no
access to the needed data / poor
data availability; poor data
integrity; poor data
confidentiality, and others)
• Lack of a data management plan
• Non-management of data per the
data management plan
• Non-preservation of digital data
into digital “forever”
• Publishing Mills, Conference
Mills, Etc.
• Fake Reviewers (including
Impersonators of Persons in the
Field), Fake Double-Blind Peer
Reviews
• ...and (many) others
7

8. Some Highlights from Prior Slides
• There are many complex steps in the research sequence, and errors
may be introduced at any step.
• Every member of a team matters. Each has to hold up his/her own
responsibilities, and each has to hold up each other effectively (even
if this means contravening social conventions to call out others clearly
and with respect).
• Leadership matters.
• Review occurs in the present; it occurs forwards and backwards in
time. As more up-to-date techniques and technologies emerge, prior
works can be checked against newer knowledge with more cutting-
edge approaches. Truth outs.
8

9. Academics and Fraud
• One survey study examined actions that distort scientific knowledge
but that do not include plagiarism (using others’ ideas without
crediting them) and other forms of research misconduct.
• A minority of the respondents, 1.97%, admitted to have “fabricated,
falsified or modified data or results at least once” and “up to 33.7%
admitted other questionable research practices” (Fanelli, May 29,
2009).
• And: “14.12%” of survey respondents said that their colleagues
engaged in data falsification and “up to 72% for other questionable
research practices” (Fanelli, May 29, 2009).
9

10. Intrinsic and Extrinsic Risks for Researchers
Intrinsic Risks
• Personal ego
• Self-deception
• Particular Dark Triad personality
dimensions
• Dated skills, especially in a
context of high aspiration and
high imagination.
• Dated knowledge of standards
Extrinsic Risks
• One’s social network (depending
on who is in it and what they
think and how they behave)
• Poor leadership (micro, meso,
and macro levels)
• Poor or de-toothed oversight
• Budget and financing drought
• A lack of integrity culture
10

11. Intrinsic and Extrinsic Risks for Researchers
(cont.)
Intrinsic Risks
• Loose handling of assertions
• Likewise, loose handling of
research data and image artifacts
• An inability to handle external
pressures with reasoned
responses
Extrinsic Risks
• Excessive pressure to perform
• Available technologies for digital
image editing
Some cultures that involve
ignoring image manipulations
11

12. Intrinsic and Extrinsic Risks for Researchers
(cont.)
Intrinsic Risks
• A personal resistance to
reaching out to others for advice
and support
• A lack of sufficient internalized
trained professional ethics
Extrinsic Risks
• Monetary incentives
• Reputational incentives [being a
“star” researcher may mean a
heightened likelihood of
research misbehavior but a
lower likelihood “to be caught
than average scientists”
(Lacetera & Zirulia, 2009, p.
568)]
12

13. 13

14. Positive Control on Digital Image Editing in
Research Context
Self-Deception
• Misunderstanding the image
data
• Falling for spurious data
Other-Deception
• Misreporting the image data
• Emphasizing particular parts of a
digital image that results in a
lack of a “pure data stream”
(Anderson, Jan. 21, 1994)
14

15. Session
Research is a critical part of work and study in higher education. There
are a raft of professional standards about how research may be
conducted, to protect humans (and animals) in the processes…and to
ensure accuracy and non-bias in the work. There are rules for data
handling, so as to avoid potential human mistakes and / or
manipulations in their data handling, data cleaning, and other efforts.
(You can engage in data exploration but need to avoid p-hacking or
seeking statistical significance from the data and reverse-engineering a
hypothesis post hoc as if it were an a priori one. You can clean data by
dropping outlier data points, but you cannot actively work to skew
data.)
15

16. Session (cont.)
• Some research studies use various visuals in their work: photos (macro
and micro), scans, screen captures, video stills, and others. The visuals are
affected by the state of the analog world, the capturing devices and
technologies, the parameter settings on the various devices, the skill of the
persons, and other factors. By the time the researcher or student has an
image for digital image editing, there may be various challenges: focus,
color balance, depth of field, lighting, and others. The question is: How
much more digital image editing can be done to that and still maintain
fidelity (given that the image contains research data and is research data in
most cases) and not lapse into artificiality (and potentially, fraud). How can
researchers stay as close to high-fidelity and true as possible? What are
the right approaches in an age of controlling images down to the pixels (in
raster imagery), artificial intelligence (AI)-enhanced digital image editing,
“deep fakes,” and counter measures against such (with built-in forgery
detections)?
16

17. Session (cont.)
1. Is it fair to raise the image resolution for print? (by enabling the software
to interpolate missing pixels)
2. Is it fair to change the white color balance?
3. Is it fair to change the lighting through artificial means?
4. Is it fair to zoom and crop for particular focuses/foci? Is it fair to rotate
or flip or skew / tilt / lean a visual?
5. Is it fair to remove visual information and replace with filler pixels?
6. Is it fair to move an object within the image to another location? Is it fair
to resize?
7. Is it fair to apply a diagnostic color filter to highlight particular insights?
17

18. Session (cont.)
8. Is it fair to mask a visual? This means selectively hiding some parts of an
image and selectively revealing or highlighting others.
9. Is it appropriate to compose / composite / fuse a visual or create whole
combined images from various other images, in pieces and parts? In
what visual representational contexts?
10. How much explanation should go into the descriptions of complex
imagery? How much depth?
11. Is it fair to use selective language to point to some aspects of an image
(as research data) but not others? Is it fair to avoid contravening
evidence to one’s (pet) hypothesis?
12. Is it fair to “batch process” a number of mostly similar images but with a
few that do not meet the basic parameters of the set?
18

19. Session (cont.)
13. If the researcher or team have a particular aesthetic preference for the
visuals (or a branding message preference), how much can they express
this in their digital imagery (as data)?
14. With the advent of machine learning and AI and their integration in
Adobe Photoshop 2021, how much should a researcher use these
features? Skin smoothing? Facial expression tweaking (using neural
filters)? Artificial art style transfers? Others?
• This work will use Adobe Photoshop 2021 to show some of the capabilities
of the tool and how these can be used in alignment with “true” and “non-
true.” (Various fields and disciplines may have different standards for what
edits may be made ethically in the profession.)
19

20. Self Intros
• Welcome!
• Who are you? What space do you work in? What digital images do
you handle?
• What are your experiences with Adobe Photoshop?
• What are some topics you would want addressed in this session?
20

21. Some Thoughts from the Field
• “Augmentation of digital images is almost always a necessity in order
to obtain a reproduction that matches the appearance of the original.
However, that augmentation can mislead if it is done incorrectly and
not within reasonable limits. When procedures are in place for
insuring that originals are archived, and image manipulation steps
reported, scientists not only follow good laboratory practices, but
avoid ethical issues associated with post-processing, and protect their
labs from any future allegations of scientific misconduct..”
• -- Jerry Sedgewick in “Acquisition and Post-Processing of
Immunohistochemical Images” in Signal Transduction Immunohistochemistry
(2017, p. 75; Ch. 4)
21

22. Some Thoughts from the Field(cont.)
• “…when procedures are in place for correct acquisition of images, the
extent of post processing is minimized or eliminated” (such as for
white balancing…tonal values in dynamic range…noise
elimination…bitrates…and others)
• -- Jerry Sedgewick in “Acquisition and Post-Processing of
Immunohistochemical Images” in Signal Transduction Immunohistochemistry
(2017, p. 75; Ch. 4)
22

23. Some Thoughts from the Field(cont.)
• “Concerned not so much with intentional fraud, but rather with
routine and ‘innocent’ yet inappropriate alteration of digital images,
several high-profile science journals have recently introduced
guidelines for authors regarding image manipulation, and are
implementing in-house forensic procedures for screening submitted
images.”
• -- Emma K. Frow in “Drawing a line: Setting guidelines for digital image
processing in scientific journal articles” in Social Studies of Science [2012,
42(3), 369 – 392 (on p. 369)]
23

24. Some Thoughts from the Field(cont.)
• “In journals that check figures after acceptance, 20 – 25% of the
papers contained at least one figure that did not comply with the
journal’s instructions to authors. The scientific press continues to
report a small, but steady stream of cases of fraudulent image
manipulation.”
• -- Douglas W. Cromey’s “Digital images are data: And should be treated as
such” in Methods in Molecular Biology (2013, 931, 1 – 27)
24

25. Some Healthy Practices…to Start
25

26. Pristine Master Set
• The research team maintains raw original photos and scans in the
highest resolution in a pristine master set.
• This enables reversion to the original by the research team. (Some may want
to “discard” the originals once more refined visuals are available, but that is
not advisable. Anything discarded in an irretrievable way means an
irrecoverable loss of data.)
• Enable a practical “undo.”
• This capability is somewhat mooted if something already goes to publication
in a published imageset or publication.
26

27. Proper Initial Image Capture / Acquisition
• Digital image editing can only do so much for a poorly captured initial
image (although the technologies are improving, and Adobe
Photoshop just came out with Super Resolution in March 2021, which
relies on AI to interpolate additional pixels to an image for very high
artificial-enhanced fidelity).
• Ideally, the original image should be as information-rich as possible.
• Artificial pixels are not appropriate to suggest into a research image
since the AI does not understand the context of the image, in many
research cases. Added pixels may be misleading, and these pixels
may be confused with actual data.
27

28. Provenance / Lineage of Imagery
• All visuals use have an established provenance (or lineage).
• It is clear where they came from and how they were acquired.
• There has been a clear “chain of custody.” Along the way, who can
touch and influence the visuals / image data?
• It is clear how they have been handled since they were acquired.
• It is clear if they went through any conversions or transforms or image
compressions.
28

29. Strong Foundations to Arrive at the Visual
• If visual is about a model or a hypothesis or a framework or a
concept, then the foundation for that should be solid. The theorizing
should be logical. The underlying data should be solid.
• If the visual is from underlying data, then the acquisition, cleaning,
analysis, and visual representation of that data should be solid.
• If the visual is from sourcing, that source should be solid.
• In other words, where a visual came from should offer a solid basis
for the depiction. The depiction itself should follow correct
conventions for representation. It should not be misleading or easily
mis-interpretable.
29

30. About Controlling for Clarity
• Parts of the visual should be properly labeled.
• The parts of the visual should follow image conventions so as not to
confuse users.
• Surrounding information about that visual should be accurate. This
would be the text, other visuals, titling, and captioning.
• Proper sizing measures and dimensions should be indicated as
relevant.
• Colors should be properly balanced.
30

31. About Controlling for Clarity (cont.)
• Legends should be accurate.
• The visual should be controlled for all possible intended and
unintended uses of the visuals.
• Control for sins of commission and for sins of omission. Avoid
suggestiveness. Avoid inaccurate assertions. Avoid omitting context.
Avoid outsized claims.
• Make sure all digital and informational contents are accessible (across
a range of perceptual and brain processing capabilities).
31

32. About Controlling for Uncertainty
• Uncertainty should be represented accurately. Assumptions should
be represented accurately.
• If an amount of uncertainty can be represented, that amount should
be accurate and indicated.
32

33. Understanding the Audience
• It helps to be able to understand the “interpretive lens” of the
audience and how they will consume the visuals.
• Some challenges arise with a larger audience with a wide range of
individuals of differing backgrounds and swaths of non-expertise in
the space.
• Other challenges arise when a visual is separated from the original
context and is not supported by augmenting and complementary
information.
• Or similarly, there may be challenges when the visual is consumed in a stand-
alone way even if it has not been separated from other informational
contents.
33

34. Defining “Redlines,” “Fidelity,” “Artificiality”
34

35. 1. Image Resolution and Sharpening
• Sometimes, particular details may not be clear.
• A digital image editing tool enables the addition of artificial pixels for
higher resolution and even “super resolution.”
• Ideally, raster images should have sufficient bits to map the respective
images. (In some contexts, much higher resolution is required.)
• The color should be at least 16-bit to 24-bit color (true color), per pixel.
• If something is hand-drawn with technologies or born-digital, use vector
representations for scalability without lossiness.
• Save images in non-lossy formats.
• Question: Is it legit to interpolate pixels for a low-resolution image? If so, why and
when? What if the interpolation does add artifacts? (The “smart” assumptions of
the AI behind such interpolates may result in distortions and some muddiness and
some digital artifacts.)
35

36. 1. Image Resolution and Sharpening(cont.)
• The software editing tool enables the use of AI to sharpen edges
within a certain identified “radius” of the identified edges.
• Another method may be to heightened contrast…or even change up
the hue to make a more contrastive look and feel.
• Sometimes, removing color and offering a visual in b/w or grayscale
can heightened focus on the lines / edges. It can heighten the sense
of shapes.
• Question: Is it legit to “smart sharpen” an image? Or find edges? Or increase
contrast? Or render the image in b/w or grayscale? If so, why and when?
Within what limits?
36

37. 2. White Color Balance
• For truer color, and to control against too much warmth (yellow) or
cold (blue) in imagery, photos should be balanced for more neutral
tones. Setting the correct “white” is one approach.
• For print, the specular highlights should be a little muted, and the
shadows should be somewhat lightened because of ink bleed. (These
are the “curve” adjustments.)
• Question: For a print context, can colors be adjusted and “jumped” to
represent accurately in print? (CMYK) If so, why and when? By how much?
• Question: What about using color to drive attention to particular part of an
image? In a labeled way? An unlabeled way?
37

38. 3. Artificial Lighting
• In post-production, it is possible to change various lighting effects on
an image.
• Particular focal regions may be lit more to draw the human eye.
• The midtones (the brightness and colors between the highlights and
the shadows) should be of sufficient detail for texture. (This is seen in
the histogram in Photoshop.)
• Question: Should “brightness” and “contrast” be adjusted to a research
image? If so, why and when? By how much?
• Question: Should the histogram be adjusted in a research image? If so, why
and when? By how much?
38

39. 4. Zooming and Cropping and Rotation and
Flipping and Skewing
• Sometimes, especially in fieldwork and “the wild,” it is hard to control
for photography and digital image capture.
• Sometimes, there are challenges in lab-based image captures as well.
• Sometimes, what is in-frame might be extraneous to the focus of the
researcher(s).
• Question: Is it appropriate to zoom in an image? If so, why and when? By
how much?
• Question: Is it appropriate to crop an image? If so, why and when? By how
much?
39

40. 4. Zooming and Cropping and Rotation and
Flipping and Skewing (cont.)
• Rotating an image involves changing the original frame of the image by
turning the image clockwise or counter-clockwise by various degrees.
• Flipping an image along the vertical or horizontal axes involves changing
the perspective of the original image. These move where objects were
originally, in a sense, within the context of the image.
• Skewing an image involves tilting or leaning it.
• Question: Is it appropriate to rotate an image? If so, why and when? By how much?
• Question: Is it appropriate to flip an image? If so, why and when? By how much?
• Question: Is it appropriate to skew or tilt an image? If so, why and when? By how
much?
40

41. 5. Removal of Visual Information, Filler Pixels
• Sometimes, especially in fieldwork and “the wild,” it is hard to control
for photography and digital image capture.
• Sometimes, there are challenges in lab-based image captures as well.
• Sometimes, the visual does depict what the researcher wants with
sufficient emphasis.
• Question: Is it appropriate to erase information in the visual that is
distracting? (Is it okay to remove “noise”? “Texture”?) And then substitute
something else? If so, why and when? By how much?
• Question: Is it appropriate to use filler pixels to fill in particular parts that one
has cut out? Is it appropriate to use a cloning tool? A patch tool? A spot
healing tool? Is it appropriate to make your own texture and apply it to the
visual? If so, why and when? By how much?
41

42. 6. Moving an Object / Resizing an Object
• Sometimes, especially in fieldwork and “the wild,” it is hard to control
for photography and digital image capture.
• Sometimes, there are challenges in lab-based image captures as well.
• Sometimes, objects may occlude particular points of interest.
• Perhaps artifacts may have been accidentally introduced in a photo or
a scan. Perhaps there may be other kinds of visual “noise.”
• Sometimes an object looks subjectively wrong size-wise.
• Question: Is it appropriate to select an image (Select subject? Lasso tool?
Marquee tools?) and cut it out of the picture and replace it with an alpha
channel or an empty background? Move its location? Change its size? If so,
why and when? By how much?
42

43. 7. Diagnostic Color Filters (for Analysis)
• Digital means have been used for various analyses of the images.
• Some of these means may leave residuals on the current imagery.
• Question: Do you have to return to an original image and use that, or can you
use the digital image that may have residual layers or tinting or other effects?
If so, why and when? By how much?
43

44. 8. Masking (Selective Hiding; Selective
Revealing)
• Other digital image editing enables driving human visual focal
attention to parts of an image (via masking or hiding parts of an
image, via blurring, via “feathering” as a type of blur)…
• Masking involves the uses of layers to applied different filter, lighting, color,
and other effects.
• Blurring serves to “hide” details of information and drive attention elsewhere.
• Question: Is masking appropriate?
44

45. 9. Compositing / Combining / Fusing
• A combination of capabilities using layers enables emplacement of
pieces and parts of digital snippets to make wholly new (appearing)
visuals.
• Compositing generally refers to combining pieces and parts of multiple
images to create a semi-coherent / coherent new whole.
• Question: Is compositing / combining / fusing legit? If so, why and when? In
what contexts? (Maybe in the depiction of fictional or imagined scenarios?
In particular computational image analysis sequences?)
45

46. 10. Explanatory Depth re: Complex Imagery
• Some images and visuals are highly complex.
• The details in such are always finite and limited.
• Sometimes, it may take several visuals to explain a concept.
• Making a visual explanatory, even if it is separated from the original
slideshow or paper, requires more work.
• Question: Should the researcher or research team make the effort to make
the visual clear in the slideshow / paper? If so, why and how? What sorts of
alt text should be included with each visual?
46

47. 11. Selective Explanatory Language
• A set of research visuals seem to contravene the researcher’s hypothesis.
• These findings may sink the hypothesis that the researcher has posited years ago and
spent years trying to explore (and maybe to support in his/her heart of hearts).
• The presentation of research is always somewhat selective. After all, not
everything can be shared. Some aspects of the research are more relevant
than others.
• Question: Is it fair to avoid contravening data? Is it fair to withhold information and
not share that when publishing? If so, why and when? By how much?
• Question: Or should the contravening data just be included in the footnotes? If so,
why? How?
47

48. 12. Batch Processing with Macros
• Automatic image processing is highly helpful in many circumstances where
there is a large number of visuals to process simultaneously…and for which
a known sequence of image handling has been designed (and expressed as
macros or as small programs).
• Automation is important for consistency, for controlling against human
error, and for efficiencies, among others.
• However, sometimes, not all images may meet the requirement that they
are of a type and meet particular criteria or have basic parameters.
• Question: Is it fair to run the whole set in a batch even if some of the images that do
not fit the criteria are included? If so, why and when? What are some other ways to
engage this issue?
48

49. 13. Aesthetics and / or Branding
• The researcher or research team may have preferences for particular
1. image aesthetics (look and feel) or
2. branding (messaging about the organization or team of work project).
• Perhaps the funding agency wants particular presentational
aesthetics and / or branding.
• Question: Is it fair change up the visuals in a research work to align with
particular aesthetics? If so, why and when? What are some other ways to
engage this issue?
• Question: With particular branding messages? If so, why and when? What
are some other ways to engage this issue?
49

50. 14. Machine Learning and AI Features
• Adobe Photoshop 2021 enables machine learning and AI features,
many of which are very smooth.
• It is possible to smooth skin.
• It is possible to change up facial expressions of people in a photo.
• It is possible to apply art styles fairly seamless from a preset work to
another.
• Question: When is it appropriate to use AI neural filtering and other features
to look better to others? Is it appropriate to use the neural filtering to change
up the facial expressions of a professional adversary to make them look
worse? If so, why and when? What are some other ways to engage this issue?
50

51. Ways to Get Found Out
51

52. 52

53. Authentication Methods
• Various disciplines have their own methods for authenticating
imagery from metadata (geotags, contextual information captured by
camera), from image forensics, from image comparisons, and other
singular and mixed approaches.
• Failing authentication is one way to be found out.
• It is one way to rouse the ferrets.
53

54. Ways to Get Found Out…by People
• You can tell on yourself…
• You can tell on yourself with what you assert (privately and publicly).
Mistruths, contradictions, and slippage can be revelatory.
• You can tell on yourself with the digital / digitized imagery that you
shared under your name.
• By being in the “chain of custody” and vouching for the provenance of
the information, you are affirming the apparent validity of the
contents.
54

55. Ways to Get Found Out…by People (cont.)
• Your colleagues can tell on you. Colleagues are competitive, and they
are on the lookout for fumbles.
• Research and publishing are gauntlets. People check each other out
and check out each other’s works.
55

56. Ways to Get Found Out…by Image Forensics
• Your imagery can tell on you.
• Imagery is multi-dimensional and complex. It is revelatory in ways that most are not
aware.
• There are a number of image forensics tools for automated identification of
edited digital images (especially in 2D and some now in 3D), including the
edits that may indicate fraud.
• There are physics-based methods (Riess, 2017). Light falls a certain way
based on universal rules. Reflectance as a normalized phenomenon can be
used to identify anomalies (Riess, Pfaller, & Angelopoulou, 2015).
• There are programs that can identify cameras that were used to take
images, identify the social network platform an image came from, and the
software used to upload the images (from the images alone) (Giudice,
Paratore, Moltisanti, Battiato, 2017) and so reconstructing the history of an
image.
56

57. Ways to Get Found Out…by Image Forensics
(cont.)
• Some technologies enable the identifying of tampered regions of a digital
image.
• One approach involves using “linear local features” to identify “copy-move forgery
detection” (Kuznetsov & Myasnikov, 2017, p. 305).
• Various wavelet analyses approaches are also used to identify regions of
interest for anomalies.
• There are programs that detect anomalies in the color gamut, in grayscale,
in gamma ranges, and others.
• Histogram normalization (in distribution) can bring out anomalies in brightness /
darkness.
• There are ways to separate out colors in RGB and others that enable
identification of anomalous regions.
57

58. Ways to Get Found Out…by Image Forensics
(cont.)
• There are validation approaches:
• Another approach is an “image hashing” one with “compressive sensing” to
validate (Sun & Zeng, 2014).
• Watermarking (an older embedded technology) is another common
approach.
• Digital signatures is another active method for forgery detection.
• Blockchain technologies are being used as well to establish “original”
works as one-of-a-kind. By elimination, all other unvalidated
versions are then off-true (and fakes).
58

59. Ways to Get Found Out…by Image Forensics
(cont.)
• The visuals on the Web and Internet are mapped, and it is possible to
reverse-image-search them (to the tune of tens of billions).
• Such setups can be done in all fields for published research. This provides a
sense of institutional memory, against which new works may be compared
computationally and non-computationally.
59

60. Ways to Get Found Out…by Image Forensics
(cont.)
• A number of digital image manipulation detection web services are
coming online according to a number of academic research articles.
• This means that publishers and others may put into place fairly efficient
vetting.
• This also means that people do not particularly need to have a special interest
in you to find you out. The computational costs will be minimal to acquire
that information.
• Retouches, doctoring, and other digital image manipulations can be
eminently seeable and empirically established.
60

61. Older Image Forensics to Check Images
• The Office of Research Integrity has some Image Forensics tools for
researcher use:
• Forensic Droplets
• Advanced Forensic Action set
• The above is apparently out-of-date and uses a much older version of
Adobe Photoshop.
• Other image forensics tools have long replaced these.
• These are still available by email to the organization, according to the
current website.
61

62. Ways to Get Found Out in 3D
• There have been advances in the 3D space to test “if a 3D point cloud
generated from a LiDAR scan has been subsequently manipulated”
(for potential usage in law enforcement) (Ponto, Smith, & Tredinnick,
2019, p. 101)
• The methods include identifying “discontinuities on octant boundaries”
(Ponto, Smith, & Tredinnick, 2019, p. 104), density gradient analysis (but
complicated due to combination of multiple such scans into a composite for
the scenes) (p. 103), and “spherical sampling” (p. 105)
62

63. Closing Thoughts
63

64. So…What is Arrayed Against the Artificial?
• Such artificial imagery may be caught in any phase: research,
presentation, peer review, publication, post-publication.
• Where there’s one misappropriation on the surface, people will look for a lot
more underneath.
• There is a “market” for calling out others, in part, to keep the research stream
more accurate and to protect the discipline.
64

65. So…What is Arrayed Against the Artificial? (cont.)
• Policy regimes have been set up against research fraud with severe
penalties, as a deterrence against such actions.
• The social norms around this practice can be unforgiving.
• For some, engaging in fraud is crossing a Rubicon.
• Research integrity is built into various curricula.
65

66. So…What is Arrayed Against the Artificial? (cont.)
• There are image recognition technologies powered by artificial
intelligence (for exact searches, for similarity ~ searches, for discovery,
for image curation, and others).
• There is computational memory of what has already been shared in
the “research stream” (published research), in the WWW and
Internet, in repositories and in referatories, and others, etc.
• There are known patterns of image fraud that have been identified
and mapped. There are “tells” or “indicators” that show
manipulations.
66

67. Range of Negative Outcomes Possible from Data
Falsification or Manipulation or Fabrication
Macro Level
• Public confidence in publicly-
funded research can be at stake.
• Grant funding can be at stake.
• A discipline may be harmed.
• University reputation may be
harmed.
• Grant funding agency reputation
may be harmed, etc.
Micro
• Published papers may be retracted,
and various digital libraries and
archives keep public records of
such retractions.
• Discovery of the data falsification
may be career ending.
• Professional reputations may be
ruined.
67
Meso Level

68. Some New Thinking and Precautions
• Get out of the mindset of absolute refinement of images and some
“perfection,” because that can lead to image edits that can be
negative to image integrity (and turn the image to artificial). [Don’t
use selfie standards for your research imagery.]
• Support your discipline in engaging a norm of the real vs. the prettified faux
real. Create a new social-professional norm.
• If the image capture did not work the first time, do it again the right
way.
• Be aware of all the implications of digital image editing on each
visual…and control for that. Be careful of processes masked in batch
processing and sequences.
68

69. Thinking and Acting Strategically for “Long
Term” Considerations
• This slideshow’s scenario has a “discrete-time” approach.
• Over time, however, a solid pristine imageset as data can be used
potentially for other exploratory research and analysis. This follow-on
analysis may be done with new techniques and new technologies.
• Having a raw pristine set of digital imagery may be informative for other as-
yet not-conceptualized analyses.
• Capturing such information in all likelihood involved much investment
of time, equipment, resources, expertise, and other inputs. Not
preserving a pristine master set of visuals would be a losing (or
“dominated”) strategy in game theory.
69

70. Some Pseudo- “Defenses” / “Excuses”
• Intentionality:
• I wasn’t intending to mislead
• I was trying to convey an accurate view in the new medium or modality
• I was trying to clean the image
• I was trying for a more aesthetically pleasing image
• Insufficient training: Nobody told me
• Lack of control on recipient understandings:
• I can’t control how information is received and interpreted
• Others
70

71. Starting to Drift?
• If you believe that there is perfect image data and that some edits will
get you there…
• If you feel like putting a thumb on the scale…
• If you believe that you can do this and slide under the radar…
• If you feel yourself starting to drift towards image manipulation…
• …what should you do?
• …how do you get back to true?
71

72. References
72

73. References
• Anderson, C. (1994, Jan. 21). Easy-to-alter digital images raise fears of
tampering. Science, 263(5145), 317 – 318.
• Fanelli, D. (2009, May 29). How many scientists fabricate and falsify
research? A systematic review and meta-analysis of survey data.
PLoS ONE 4(5): e5738.
https://doi.org/10.1371/journal.pone.0005738.
• Giudice, O., Paratore, A., Moltisanti, M., & Battiato, S. (2017,
September). A classification engine for image ballistics of social data.
In International Conference on Image Analysis and Processing (pp.
625-636). Springer, Cham.
73

74. References (cont.)
• Kuznetsov, A., & Myasnikov, V. (2017). Using efficient linear local
features in the copy-move forgery detection task. In International
Conference on Analysis of Images, Social Networks and Texts (pp. 305-
313). Springer, Cham.
• Lacetera, N., & Zirulia, L. (2009). The economics of scientific
misconduct. The Journal of Law, Economics, & Organization, 27(3),
568 – 603.
• Ponto, K., Smith, S., & Tredinnick, R. (2019). Methods for detecting
manipulations in 3D scan data. Digital Investigation, 30, 101-107.
74

75. References(cont.)
• Riess, C. (2017, September). Illumination analysis in physics-based
image forensics: A joint discussion of illumination direction and color.
In International Tyrrhenian Workshop on Digital Communication (pp.
95-108). Springer, Cham.
• Riess, C., Pfaller, S., & Angelopoulou, E. (2015, September).
Reflectance normalization in illumination-based image manipulation
detection. In International Conference on Image Analysis and
Processing (pp. 3-10). Springer, Cham.
• Sun, R., & Zeng, W. (2014). Secure and robust image hashing via
compressive sensing. Multimedia tools and applications, 70(3), 1651-
1665.
75

76. Image Fidelity in
Research
• What does “image fidelity” in your
area of research look like, and
why?
• How do you achieve the proper level
of image fidelity for professional
practice?
• Where are the risks of potentially
lapsing into or choosing
artificiality?
• What are the “best practices” to
avoid image manipulation?
76

77. Some Versioning Notes and Caveats
• Versioning Notes: A first draft of this was shared on SlideShare in early March
2021. Since then, I have reviewed some more literature and added more on-
ground complexity. I updated the early version by replacing it on SlideShare. The
one online will not be the final version because I am always updating up until the
moment of presentation at which point I lock in that slideshow. Behind the
scenes, I am still learning about the topic though.
• I understand the benefits of having a single source for an “authoritative” copy and will likely
update using the final copy used in the presentation.
• Caveats: This is a first run at the topic only and does not actually represent either
the full capabilities of the digital image editing software nor the complexities of
the academic research space in particular disciplines/domains nor the different
types of digital imagery used in research. This does not touch on deeper image
forensics capabilities either, which can be very sophisticated in-field for various
domains.
77

78. Presenter Information and Contact
• Dr. Shalin Hai-Jew
• Instructional Design / Research / Training
• Academic and Student Technology Services
• ITS
• Kansas State University
• shalin@ksu.edu
• 785-532-5262
• All contents including visuals are by the presenter except for the cited
published sources, which are credited.
• CHECK 2021 is the Conference on Higher Education Computing in
Kansas.
78