Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Even lexing ruby for highlighting is really hard. I had to use so many lookahead hacks to get the lexer in rouge working: https://github.com/jayferd/rouge/blob/master/lib/rouge/lexer...

I learned a ton about ruby when implementing that. Did you know that this totally works:

    my_method <<-ONE, <<-TWO, <<-THREE
       text of one
    ONE
       text of two
    TWO
       text of three
    THREE
And all the % delimited strings are super hard.

(also note that pygments chokes on my perfectly valid ruby file, at `%r(\\\\)`)



> (also note that pygments chokes on my perfectly valid ruby file, at `%r(\\\\)`)

Pygments was actually better at lexing ruby at one point but the regular expressions were too complex and some other bug fixes broke other stuff. I think at the moment it's good enough.

But yeah, I was in the same boat. I learned Ruby for Pygments.

//EDIT: vim does considerably worse on that file btw.


Yep. It's surprising how many things can be missing from a lexer and still be "good enough" for most people. I don't know anybody who's actually used that heredoc-stacking thing.

:D I guess you're the author of all those frustrated "wtf ruby" comments I found in the pygments lexer, then :). I seriously doubt I could have done it at all without pygments as a reference.

My vim does fine on that file: http://imgur.com/VSYr4aa . Maybe I'm using a more recent version...? It misses some of the `end` keywords, but that's just a quirk that it has ;)

Another example that bit me - this took the longest, I think:

    foo = 10
    foo %(2) # call method foo with string "2"
    foo % 2  # the value of foo modulo 2
    % 2      # the string "2"


These are the kinds of things that make me appreciate s-expressions so much more. It's crazy how programming languages like Ruby or C can have ridiculously complex syntax, and LISP can express all of that an then some with just three or four simple grammar rules.


For non-rubyists, what does this example do? Are those labels?


It's a way of writing strings without having to escape things or having line breaks mess things up (a heredoc, if you're familiar with that terminology). For instance,

    <<-END
      this is some text
    END
will give you a string containing " this is some text\n". END can be anything.

It's particularly useful in metaprogramming, since you can maintain formatting, and you don't have to escape " or ' if they happen to show up in your code.

What's interesting in the OP's example is that you end up passing three strings " text of one\n", " text of two\n", " text of three\n". I had no idea that would happen, though I have seen things like

    define_method <<-RUBY, __FILE__, __LINE__ + 1
      some_code
    RUBY
so it makes sense. Ruby treats anything on the same line after the comma as separate arguments, so even though __FILE__ comes after <<-RUBY and before RUBY, it really appears after the final RUBY to the interpreter.


That's ruby's heredoc syntax: http://en.wikipedia.org/wiki/Here_document

I had no idea it could be nested like that and I've been using ruby for 8 years. Crazy!


Holy cow, 10 years for me and I've never even thought of nesting heredocs.

Some days I wish Matz would formalize the grammar so it is sane.


As another non-rubyist with too much perl and bash under my belt, I'm guessing it's heredoc - the <<-FOO syntax means "grab the lines following until you see FOO alone on a line, shove it all in a string, and pass it as this argument (in bash it passes a filename that represents half of a pipe which has been sent the content).




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: