A discussion of the future of AI (artificial intelligence) programs in lens design. The weaknesses of deep learning AI programs compared to knowledge based AI programs are illustrated with many examples from lens design and also chess.

1. The future of AI in Optical Design
Dave Shafer
David Shafer Optical Design
Fairfield, CT 06824
203-259-1431
shaferlens@sbcglobal.net

2. “Deep learning” neural network AI programs are being used in facial
recognition and other pattern recognition activities and have had very much
better success as a chess playing program than old style knowledge based AI
programs. Because of this I think it is very likely that in the future automated lens
design will be handled by neural network AI programs and knowledge based
programs AI may become obsolete and abandoned.
But oddly this will make human lens designers more valuable, at least in the
near term, and I have many design examples that will show this. If the only input
to a neural network program are the rules of chess, or Snell's law and ray tracing
codes for lens design as well as the specs on a desired design, then that same
input becomes very important in a way that it is not in the old knowledge based
programs. Some examples here will show this – how an incomplete rule set as
well as a needlessly restrictive set can cause deep learning AI programs to give
odd and/or disappointing results. They may always need human guidance.

3. Before looking at AI in lens design let us first
consider AI in chess. In a different universe
Homer Simpson might be a math wizard. But
in our universe he is an idiot. How would he
do playing chess against a computer program,
specifically a deep learning AI one?

4. Imagine a robot connected to a powerful deep learning AI chess playing
program that we have just created.

5. How our chess playing robot does against Homer
Simpson will depend strongly on the set of rules of the
game given to it and the possibilities for play that
follow from that. This is also true in lens design.
The key to the outcome is how complete the rule set
is. For example, Homer would be used to a very
important rule, from having played other games, that
the computer program has to be explicitly told.
That rule is that the chess
players can only move their
own pieces, not also those
of their opponent. So very
obvious that it might not
be in the rule book given to
the program.

6. Here is a classic and brilliant chess
puzzle from 1974. The problem is for
White to checkmate in just one move.
A quick study will show that this is
impossible. A long study will show the
same thing. An AI deep learning
computer program will almost certainly
be stumped by this. The solution
depends on a very minor chess rule that
you would probably never think of
applying to this situation.
The board and pieces as shown here
are a completely legitimate arrangement
but it is shown in a very deceptive way,
which is the key to the answer.

7. In setting up the initial board it is
required that the bottom right hand
square space be white.
right wrong
Board is
rotated 90
degrees from
correct position

8. Here is the board shown
earlier. The lower right hand
square is black, not white, so
this view is very misleading, but
still legitimate.
Here is the “correct” (i.e., not
misleading) view, with a 90 degree
rotation. Now the bottom right
hand square is white.

9. With this view everything is different. The pawn can be advanced by one square
and promoted to a knight, which gives an immediate checkmate in just one move.
The puzzle
depends on
showing the
board mid-
game, when
you would
probably not
notice the 90
degree board
orientation.

10. But a human, even a cartoon human,
working on this puzzle might decide that
there is something fishy going on here,
where it is easy to show that there is no
solution possible and yet we are told to
find a solution. A computer would not
think that way or be suspicious unless it
had been explicitly told to check the rule
book when all else fails.
Most test situations are not a contest
between you and the test material (here
a chess puzzle). They are instead a
contest between you and the person
who created the test – here a very
sneaky person.
Most (or maybe all) AI programs are
unaware of this particular human
element. A sneaky diabolical puzzle
creator is a higher level concept outside
of the AI program’s rigid rule book
orientation.

11. Here is a simple Turing test, based on human psychology, to
distinguish between a computer and a person. It is based on an
awareness of how people think. The next slide shows a whole
bunch of computer generated 5 digit random numbers. But
suppose instead these random numbers were all generated by
a person. How can you easily and quickly tell which it is?
Answer – you quickly scan down through the columns of
numbers looking for some that are mostly zeros, like 90003 or
00006, or repeated numbers, like 77773. These stand out
visually from the background and are easy to spot. If you see
none or very very few of these then you know the list was
generated by a person. Few people will include a number like
90003 or 10001 in a list they generate because they don’t think
these numbers “look random”. So here is a Turing test that no
AI program would ever “think” of generating, because it is
based on human psychology. It is a knowledge based test idea.
Turing

12. 87736 79526 39965 19969 89741 69822 05135 41282 00340 39419 17169 67703 58501 77654 89306 15768 45873 45836 18868 49765 00841
35060 13818 02049 57589 49018 44329 18352 10168 11815 19343 28101 01551 00775 88133 39724 37703 04258 66867 60966 26637 04473
90172 44203 42480 85289 87691 56748 14535 65046 85651 76173 41344 54370 33424 10613 29748 38987 55088 44083 58953 87528 25991
06788 69590 84352 74313 70441 77779 87351 07628 68566 14189 10762 64468 41820 88104 32987 90885 08300 76483 29398 42009 58855
38024 30151 08621 77405 52224 03003 68704 86244 83447 61118 64012 16360 69978 95884 01631 02745 38094 34725 80543 04706 41121
23912 43184 51181 23498 36374 52013 11047 09842 04516 91068 88437 19005 78027 92534 43049 03143 06692 88301 35460 11773 48208
86529 97440 80840 00996 73212 27260 46773 62008 38735 10011 85558 09546 29444 62910 76788 11074 24201 05616 92737 56026 35952
55543 12414 13622 38288 48171 60629 15679 17004 82358 72058 64425 46669 90654 80300 65311 92180 34613 31942 84822 35331 25106
25624 33292 26372 67632 76099 72713 60574 32631 03293 49553 79786 50798 46374 52562 99007 33395 02808 95588 57939 17561 92426
25392 97556 36626 46483 54455 61661 15428 21725 18794 23456 90616 39446 60772 37095 55972 49555 36916 14256 32717 19635 50721
25696 64828 49011 10471 27817 39347 07804 12046 14231 31694 01001 46736 92314 75983 57516 34054 84780 60329 44247 79547 56233
47166 05147 57613 64736 18555 25281 19306 38372 46500 19889 25156 76805 88258 48449 02975 77843 06651 53742 30676 89003 15383
25910 44080 84070 49348 91521 69731 64661 59942 27633 82708 86789 35381 00854 89439 12422 46007 31403 35511 86049 14604 82179
38482 36240 11711 72815 45699 02210 75933 18748 17105 85383 59253 95277 33051 25948 97347 26626 27113 04431 04893 00009 43902
68248 95886 04690 21876 83276 28951 96046 01153 10409 77654 84629 56325 07125 25815 50232 54268 54043 87818 80946 36956 06493
53929 37226 29272 71073 60605 40954 71823 07212 85805 89499 78752 53736 69306 94781 20548 80425 58014 47334 31860 64580 76276
05366 44310 92585 21192 08826 44307 86467 82930 18461 55855 55461 90035 79884 48369 72570 79746 12631 69488 76132 93304 24228
02885 96377 96266 49252 51080 06992 23820 13392 43254 01685 51094 22514 68061 95745 88469 76722 31440 52606 01894 09419 39427
35409 58197 51375 05968 83564 95397 19781 85732 14314 18015 32322 31925 41951 24828 17718 33232 47888 89071 03822 63420 16822
28778 70174 32942 27518 63902 88167 33871 85012 35586 65620 48999 22170 01478 60838 09657 26256 56043 21680 78522 90487 75873
49618 57430 90603 90002 88796 20192 86424 75220 40102 70402 48416 76380 00219 24253 09249 51717 25698 18349 40366 43155 22013
42513 53541 71624 57628 97608 27442 57368 03056 78341 59193 65263 94924 81080 40941 75123 25036 36667 61411 77018 54433 68229
32475 91585 52456 73987 40711 15338 60812 35078 23968 90983 82631 48357 72924 02731 96532 84216 57656 96609 95686 85575 22642
13217 54913 33310 56412 05972 19445 36321 82560 46987 63027 30698 49065 23572 29976 64737 67285 32901 35021 75505 82270 71074
96112 90401 68076 47179 92235 53718 48070 22356 46222 78207 40254 73856 32567 21636 27568 84107 10237 88359 53206 54775 80482
55870 40930 45829 69652 26375 40378 39597 37004 03345 97384 29836 07522 93085 49281 35790 29722 31295 45221 41256 58468 34632
91485 58640 62785 77271 21795 18027 58671 92471 92627 81240 66907 99134 03522 80903 95875 53369 42403 84500 49363 25057 55881
83446 59386 99407 44573 07141 43991 03526 83412 62035 90075 57977 66959 95694 61192 53341 79719 85859 30371 51784 91447 54507
37927 13613 77467 06919 74271 48845 63983 95737 87723 93766 98387 10030 17329 12492 44577 89355 50668 23535 03267 62820 06666
51365 57301 13145 43412 02971 41963 65727 90963 69422 40291 02701 09167 52433 03390 02003 70465 15928 95743 01432 67709 62597
65526 87713 46977 35464 39918 10559 65377

13. 87736 79526 39965 19969 89741 69822 05135 41282 00340 39419 17169 67703 58501 77654 89306 15768 45873 45836 18868 49765 00841
35060 13818 02049 57589 49018 44329 18352 10168 11815 19343 28101 01551 00775 88133 39724 37703 04258 66867 60966 26637 04473
90172 44203 42480 85289 87691 56748 14535 65046 85651 76173 41344 54370 33424 10613 29748 38987 55088 44083 58953 87528 25991
06788 69590 84352 74313 70441 77779 87351 07628 68566 14189 10762 64468 41820 88104 32987 90885 08300 76483 29398 42009 58855
38024 30151 08621 77405 52224 03003 68704 86244 83447 61118 64012 16360 69978 95884 01631 02745 38094 34725 80543 04706 41121
23912 43184 51181 23498 36374 52013 11047 09842 04516 91068 88437 19005 78027 92534 43049 03143 06692 88301 35460 11773 48208
86529 97440 80840 00996 73212 90002 27260 46773 62008 38735 10011 85558 09546 29444 62910 76788 11074 24201 05616 92737 56026
35952 55543 12414 13622 38288 48171 60629 15679 17004 82358 72058 64425 46669 90654 80300 65311 92180 34613 31942 84822 35331
25106 25624 33292 26372 67632 76099 72713 60574 32631 03293 49553 79786 50798 46374 52562 99007 33395 02808 95588 57939 17561
92426 25392 97556 36626 46483 54455 61661 15428 21725 18794 23456 90616 39446 60772 37095 55972 49555 36916 14256 32717 19635
50721 25696 64828 49011 10471 27817 39347 07804 12046 14231 31694 01001 46736 92314 75983 57516 34054 84780 60329 44247 79547
56233 47166 05147 57613 64736 18555 25281 19306 38372 46500 19889 25156 76805 88258 48449 02975 77843 06651 53742 30676 89003
15383 25910 44080 84070 49348 91521 69731 64661 59942 27633 82708 86789 35381 00854 89439 12422 46007 31403 35511 86049 14604
82179 38482 36240 11711 72815 45699 02210 75933 18748 17105 85383 59253 95277 33051 25948 97347 26626 27113 04431 04893 00009
43902 68248 95886 04690 21876 83276 28951 96046 01153 10409 77654 84629 56325 07125 25815 50232 54268 54043 87818 80946 36956
06493 53929 37226 29272 71073 60605 40954 71823 07212 85805 89499 78752 53736 69306 94781 20548 80425 58014 47334 31860 64580
76276 05366 44310 92585 21192 08826 44307 86467 82930 18461 55855 55461 90035 79884 48369 72570 79746 12631 69488 76132 93304
24228 02885 96377 96266 49252 51080 06992 23820 13392 43254 01685 51094 22514 68061 95745 88469 76722 31440 52606 01894 09419
39427 35409 58197 51375 05968 83564 95397 19781 85732 14314 18015 32322 31925 41951 24828 17718 33232 47888 89071 03822 63420
16822 28778 70174 32942 27518 63902 88167 33871 85012 35586 65620 48999 22170 01478 60838 09657 26256 56043 21680 78522 90487
75873 49618 57430 90603 8796 20192 86424 75220 40102 70402 48416 76380 00219 24253 09249 51717 25698 18349 40366 43155 22013
42513 53541 71624 57628 97608 27442 57368 03056 78341 59193 65263 94924 81080 40941 75123 25036 36667 61411 77018 54433 68229
32475 91585 52456 73987 40711 15338 60812 35078 23968 90983 82631 48357 72924 02731 96532 84216 57656 96609 95686 85575 22642
13217 54913 33310 56412 05972 19445 36321 82560 46987 63027 30698 49065 23572 29976 64737 67285 32901 35021 75505 82270 71074
96112 90401 68076 47179 92235 53718 48070 22356 46222 78207 40254 73856 32567 21636 27568 84107 10237 88359 53206 54775 80482
55870 40930 45829 69652 26375 40378 39597 37004 03345 97384 29836 07522 93085 49281 35790 29722 31295 45221 41256 58468 34632
91485 58640 62785 77271 21795 18027 58671 92471 92627 81240 66907 99134 03522 80903 95875 53369 42403 84500 49363 25057 55881
83446 59386 99407 44573 07141 43991 03526 83412 62035 90075 57977 66959 95694 61192 53341 79719 85859 30371 51784 91447 54507
37927 13613 77467 06919 74271 48845 63983 95737 87723 93766 98387 10030 17329 12492 44577 89355 50668 23535 03267 62820 06666
51365 57301 13145 43412 02971 41963 65727 90963 69422 40291 02701 09167 52433 03390 02003 70465 15928 95743 01432 67709 62597
65526 87713 46977 35464 39918 10559 65377

14. Homer has tunnel vision and cannot think
of anything outside of his present situation
and that limits what he can achieve.
Computers have a very hard time “thinking”
outside the box if they are too constrained
by a set of rules or alternately are working
with an incomplete set of rules. Let us
move now over to lens design and see this.
There are many situations in modern lens design programs where how the merit
function is defined and how the initial configuration data is specified will
needlessly restrict what designs can be found by the optimization program. This
is where a human lens designer or a very good knowledge based AI program can
find some designs that a deep learning program can never find. Let’s look at how
a deep learning AI program will fail when it lacks at least some basic knowledge.

15. A monochromatic design with a 90 degree field, f/1.0, 5.0 mm f.l. Many
aspherics. Diffraction-limited over the field at .55u Well corrected for distortion
and no vignetting. The key to this kind of design is many aspherics, very long
length compared to focal length and – most important of all – an intermediate
image.
There is no equivalent high performance design without an intermediate
image that can cover a 90 degree field at f/1.0 - that is a key part of this amazing
design type.

16. Suppose you gave an AI program the task of finding a good design with the same
specs, of 90 degree field and f/1.0 with no vignetting, and some focal length value. You
might choose 10 mm focal length. No matter how long the AI program searches it can
never find a design of this type. Why? Because with an intermediate image the focal
length sign changes and you should have asked for a -10 mm focal length. Few
designers would think of that and a deep learning AI program certainly would not. The
best plan is to correct for the square of the focal length or the absolute value, to cover
all + or – possibilities. Then AI could find an intermediate image design.

17. Spherical aberration between the
doublets redistributes the energy
Collimates the
aberrated rays
Intermediate focus gives a design with much
better energy capture and intensity uniformity
Gaussian beam
to flat top design

18. Spherical aberration between the
doublets redistributes the energy
Collimates the
aberrated rays
In controlling the output ray distribution during optimization, the signs of the ray heights
are flipped in the bottom design compared to the top design, because of the intermediate
image. An AI program will never find the very good bottom design if given the ray distribution
targets of the top design. All the target value signs would need to be switched.

19. Here is a good
design from 1934 by
Schmidt, where the
usual Schmidt
aspheric plate is
replaced by an
afocal triplet
without an aspheric
Reference design, f/1.0, 10 degree full field, for comparisons
BK7 lenses
Spherical
mirror

20. It turns out that there is a
variety of three lens
solutions, all the same
glass, and all spherical
surfaces. This one has
much better performance
than our reference design.
Suppose you give an AI
program the task of looking
for other designs with
three front lenses and a
spherical mirror, looking for
the best solution.

21. The best design of all,
with three lenses, is this
one here – where 2 of the
3 lenses are seen in double
pass. It is related to our
original reference design,
pioneered by Schmidt.
Despite the similarity to our reference design (positive, negative,
positive lenses, no meniscus lenses, no lens thickness sensitivity), no
“automatic design” program is ever going to find this new solution – of
using the double-pass idea - from the reference design starting point.

22. From the computer’s
point of view, and also
that of the AI program,
this is a design with 5
lenses, not 3 – because
that is what the light
rays see. But physically
it is just 3 lenses. An AI
program will never find
this high performance
solution if it only
understands 3 lenses to
mean what the light rays
see.

23. A beamspliiter used in converging light has aberrations – spherical aberration, coma, and
astigmatism – which degrade the transmitted image quality. Shenker and Rayces both
independently came up with this very clever insight – if you just choose the coordinate system
differently you will see that the aberrated rays are just a small off-center piece of a very fast
speed on-axis ray cone, which only has spherical aberration – in that coordinate system.
Very fast speed
axial ray cone
Piece of the
axial cone
Design with
coordinate shift

24. That same spherical
aberration can be
corrected by adding a very
weak power spherical
mirror either before or
after the beamsplitter. No
AI program would find this
clever solution. Instead it
probably would do what
human designers have
done in published designs
– added expensive cylinder,
wedge and asphericity to
the beamsplitter element
itself to fix the aberrations.
Spherical
mirror
Spherical
mirror

25. The Fulcher design is a
simple extremely high
performance focusing lens
with just 4 weak power
spherical lenses. Here it is
as a 100 mm focal length
design at f/0.7 and a
wavefront error of .006
waves r.m.s. at .55u
It turns out that there are several other good solutions with 4 lenses, including
an alternate all positive lens design and several where one of the 4 lenses is
negative. The Fulcher design is the best one by far and is not easy to find.
4 spherical lenses
with n = 1.6

26. To know about Fulcher’s OSA article from
the 1950s you would need to have a very
good memory (like this guy here) or have
read the very few articles since then that
refer to it.
A knowledge based AI program might
have a huge amount of design literature
and patents loaded into it, but deep
learning AI based on neural networks
seems to be the wave of the future – not
knowledge based AI. There is no guarantee
that a deep learning AI program would find
the superior Fulcher solution among the
other alternate design solutions.

27. A catadioptric immersion
design for lithography, with
two intermediate images.
An alternate catadioptric
immersion design for
lithography, with two
intermediate images
Both designs are state-of-
the-art in performance

28. It is highly unlikely that an AI design
program could find either of these
designs because of the unusual
configuration of lenses and mirrors and
the extremely high performance levels.
But if AI could find the bottom design,
with equivalent performance to the top
design, it would not “realize” that it is a
worthless design – and neither would
some designers. The reason? – The top
design has two reflections and the
bottom design has three, so the right/left
image orientation is reversed in the two
designs. Only the top design is
compatible with the already existing very
expensive lithographic circuit masks.

29. A conventional design configuration like this for a complex lithography lens
is ideal for a deep learning AI program, or a knowledge based program.
Lenses can be automatically added until a good solution is found. But designs
that combine lenses and mirrors, like the previous slide, are much more
complex and have to deal with ray/mirror clearances and interferences, fold
angles, odd geometries, etc. that would be much harder for an AI program to
“think” of and deal with.

30. Fresnel lenses that deviate light by large
angles, as in the outer parts of a lighthouse
lens, use either tiny mirrors or catadioptric
prism elements. They result in large ray
deviation angles with no color effects.
Sometimes the
simplest ideas would be
the hardest for AI to
discover. Here it is the
transitioning change
between the center and
outer parts to avoid
excessive color, as well
as what is shown next.

31. Dispersion effects in Fresnel elements
Lots of color some color little or no color effects
In principle you could make low or zero
dispersion sheets of plastic Fresnel prisms
No reflection One reflection
Two reflections

32. Cognitive scientist Roger
Shepard’s 1990 table top
illusion. The table tops
are exactly the same size
and shape! Yes they are!
It is an illusion to think that an
AI program that only uses deep
learning with no knowledge
based input can match human
designers.

33. So it looks like the future of AI
in lens design is one where
human knowledge and insights
will greatly expand the useful
terrain that AI programs explore
– by people directing AI away
from known dead ends and
towards regions not covered by
the “rule book” – such as
allowing a double pass of light
through a lens.Homer will probably not, however, be
one of the people best suited for this
human collaboration with AI programs.

34. One thing is certain - in the future AI will substantially reduce the
employment options in many fields. Even today young people are
becoming very concerned about that, as my next slide shows.