The document discusses improvements to error diagnostics in Ruby 3.1 through a new feature called error_highlight. It summarizes how error_highlight works, problems encountered during its implementation, and solutions to those problems. Key challenges included ensuring compatibility with error handling tests and logs, correcting incorrect underline locations, and improving support in frameworks like Rails. The feature is now expanded to support more exception types and frameworks through collaboration across the Ruby community.

1. error_highlight:
User-friendly Error Diagnostics
Yusuke Endoh (@mametter)
RubyKaigi 2022

2. 2

3. Yusuke Endoh / @mametter
• A Ruby committer working
at Cookpad w/ @ko1
• My major Ruby contributions:
• Keyword arguments
• coverage.so
• TypeProf
• error_highlight  Today's topic
3
https://ruby-puzzles-2022.cookpad.tech/

4. error_highlight
Ruby 3.1 reports the fine-grained error location
4
$ ruby test.rb
test.rb:1:in `<main>': undefined method `time' for
42:Integer (NoMethodError)
42.time { print "Hello" }
^^^^^
Did you mean? times
error_highlight

5. json is nil ? Or json[:article] is nil ?
undefined method `[]' for nil:NilClass (NoMethodError)
How useful?
To tell which of the calls failed in a line
Typical case:
5
json[:article][:author]
^^^^^^^^^
undefined method `[]' for nil:NilClass (NoMethodError)
json[:article][:author]
^^^^^^^^^^
json[:article][:author]
json is nil
json[:article] is nil

6. More useful than theory
Developer experience is more improved than expected
• Suggests "how to fix"
• Helps you reach the wrong code in the editor
6
$ ruby test.rb
test.rb:123:in `<main>': undefined method `gsuub' for ...
str.gsuub(/....../, "foobar")
^^^^^^
Did you mean? gsub! gsub

7. What will change in Ruby 3.2?
• ArgumentError / TypeError
• Ruby 3.1 supported only
NameError / NoMethodError
7
$ ruby test.rb
test.rb:1:in `+': nil can't be
coerced into Integer (TypeError)
1 + nil
^^^
Today's topic: the road to achieve them
• Rails' error page

8. Agenda
• ➔Background
• How error_highlight implemented in Ruby 3.1
• Problems and solutions
• Conclusion
8

9. Implementation overview
• 1. Record the code range in bytecode
• 2. Determine where to underline
• 3. Prints an error message
9
3.1

10. 1. Record code ranges in byte code
10
compile
foo.bar(1)
0..2
0..9
8..8 send "foo"
put 1
send "bar"
byte code
source code code range
0..2
8..8
0..9

11. 2. Determine where to underline
11
send "foo"
put 1
send "bar"
foo.bar(42)
0..9
foo.bar(42)
0..2
error
error
0..2
8..8
0..9
This approach is too naïve! Why?

12. ary.map do
…
end.selct do
…
end
Naïve code range is too wide
• error_highlight uses AST analysis & Regexps (!)
• AST does not retain a location of punctuation (e.g., period)
• Kevin Newton's parser work may improve the situation
(related to the next talk in this session!)
12
ary.map do
…
end.selct do
…
end
expected
ary.map do
…
end.selct do
…
end
naïve
Typo

13. Support various method calls
13
obj.foo += 1 obj.foo is not defined
obj.foo += 1 obj.foo returned nil
obj.foo += 1 obj.foo= is not defined
prinnt(str) nil + 1 nil[1]
obj&.foo
obj.
foo
obj
.foo
Suggestion for improvement is welcome

14. 3. Prints code with the underline
• Overrides NameError#message
• Ruby's error printer writes #message to stderr
• Ruby 3.1 released this version, but
14
class NameError
def message
super + "¥n" + code + "¥n" + underline
end
end
This approach had many problems!

15. Agenda
• Background
• ➔Problems and solutions
• Less expandable
• Wrong underline location
• Poor ecosystem support
• Conclusion
15

16. Problem 1: Less expandable
• Only NameError / NoMethodError were supported
• Why?
• Because an error message has some usages
• A. To show an error trace
• B. To test error-handling code
• C. To log an error
• error_highlight breaks B and C
16

17. Problem 1-B: Test compatibility
• Many tests in the wild checks an error message
• Changing Exception#message is incompatible
• Acceptable to change NameError#message since
few test cases check its result
17
expect { … }.to raise_error("foobar")

18. Problem 1-C: Logging compatibility
Many expect #message to return a one-line string
•"error_highlight makes it difficult to parse a log file!"
•"error_highlight makes so confusing!"
18
E, [2022-09-10T10:00:00.000000 #12345] ERROR -- : undefined method `time' for 42:Integer
42.time
^^^^^
Did you mean? times
#<NoMethodError: undefined method `time' for 42:Integer
42.time
^^^^^
Did you mean? times>
p $!

19. Solution for Problem 1
Exception#detailed_message is introduced
19
$!.message
undefined method `time' for 1:Integer
$!.detailed_message
undefined method `time' for 1:Integer (NoMethodError)
1.time
^^^^^
Did you mean? times

20. Request to framework developers
• Frameworks may want to use #detailed_message
• Instead of #message
• Example: https://github.com/rack/rack/pull/1926
• Sentry and DataDog said they would support this
• https://bugs.ruby-lang.org/issues/18438
• thanks to Stan Lo (Sentry) and Ivo Anjo (DataDog)
20

21. Problem 1 is solved
ArgumentError / TypeError are now supported
Without significant incompatibility
21
test.rb:1:in `+': nil can't be coerced into Integer (TypeError)
1 + nil
^^^

22. Problem 2: Wrong underline
Ruby's error printer "escapes" the message
22
test.rb:1:in `<main>': undefined method `gsuub' for " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥":String (NoMethodError)
" ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "")
^^^^^^
" ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "")
Wrong!
Wrong too!

23. Solution to Problem 2
Stopped Ruby's error printer from escaping
23
This simple solution took about a month! Why?

24. Why did it escape the message?
• Because of security consideration!
• "TERMINAL EMULATOR SECURITY ISSUES" [1]
• Some terminals had insecure escape sequences
• Create a file
• Input as if a user typed it
24
[1] https://marc.info/?l=bugtraq&m=104612710031920&w=2

25. Is this concern still valid?
• [1] is very old (in 2003) and also says:
• Surveyed the current situation
• Rxvt / Eterm / XTerm disabled the dangerous features
The terminals mentioned in [1]
• Gnome Terminal, Windows Terminal and iTerm2 don't support
the dangerous features
25
The responsibility should rest on the actual terminal emulator

26. Problem 2 is solved
• We agreed for Ruby to stop escaping
• Correct underline in Ruby 3.2
26
test.rb:1:in `<main>': undefined method `gsuub' for " ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥":String (NoMethodError)
" ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "")
^^^^^^
" ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥ ⃥".gsuub("", "")

27. Problem 3: Poor ecosystem support
Rack error page
27
broken

28. Solution to Problem 3
Write a patch! https://github.com/rack/rack/pull/1925
28
<pre>

29. Support Rails error page
• Rails provides a more
dedicated error page
• error_highlight provides
only Exception#message
Parse #message …?
29

30. Export error_highlight API
30
ErrorHighlight.spot(
$!,
backtrace_location:
$!.backtrace_locations[0]
)
{
:first_lineno=>2,
:first_column=>3,
:last_lineno=>2,
:last_column=>8,
:snippet=>" 1.time¥n",
:script_lines=>[…]
}

31. Problem 3 is solved
Rails error page now
shows error_highlight
The patch is merged
https://github.com/rails/rails/pull/45818
31

32. Agenda
• Background
• Problems and solutions
• ➔Conclusion
32

33. Ruby 3.2's error_highlight
• ArgumentError / TypeError
Ruby 3.1 supported only
NameError / NoMethodError
33
$ ruby test.rb
test.rb:1:in `+': nil can't be
coerced into Integer (TypeError)
1 + nil
^^^
• Rails' error page

34. Acknowledgments
• @yui-knk made the early prototype of error_highlight
• @ioquatix reported a lot of issues
• Those who joined in the discussion on tickets, PRs, or
the dev meeting
34

35. 35

36. Memory overhead of bytecode
• Approx. 3% (on scaffold Rails app)
• Record AST node_id instead of raw linenos/columns
• Simple compression
36

37. Future work / Known problems
• Support CJK full-width characters
• I don't want to do that!
• Support more exception classes
• User-defined exceptions
• Show column number
• test.rb:2:2
37

38. Does error_highlight use escape sequence?
• It can, but I have no plan
• Why?
• Terminal messages are often copy/pasted as a text
• Critical use of escape sequence will make people want to
use screenshots
• ➔ Searching by error message will be difficult
38

39. Lots of considerations and tasks
• ANSI escape code (= terminal font style)
• Gems
• did_you_mean gem (thanks to Yuki Nishijima)
• syntax_suggest gem (thanks to Richard Schneeman)
39
$!.detailed_message(highlight: true)
undefined method `time' for 1:Integer (NoMethodError)
1.time
^^^^^
Did you mean? times