How are negative int/float literals handled during lexing/parsing?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

How are negative int/float literals handled during lexing/parsing?

Johan Tibell
Hi!

I'm trying to read the compiler code (mostly because I'm curious) and right now I'm trying to understand the handling of negative integer/float literals.

In token.rs we have:

    pub enum Lit {
        ...
        Integer(ast::Name),
        Float(ast::Name),
        ...
    }

So at this stage these literals are still just (interned) strings. In lexer/mod.rs we can see that the lexer doesn't consume any plus/minus sign, so the above tokens don't contain them.

In ast.rs we have:

    pub enum Expr_ {
        ...
        /// A literal (For example: `1u8`, `"foo"`)
        ExprLit(P<Lit>),
        ...
    }

    pub enum Lit_ {
        ...
        /// An integer literal (`1u8`)
        LitInt(u64, LitIntType),
        /// A float literal (`1f64` or `1E10f64`)
        LitFloat(InternedString, FloatTy),
        /// A float literal without a suffix (`1.0 or 1.0E10`)
        LitFloatUnsuffixed(InternedString),
        ...
    }

    pub enum Sign {
        Minus,
        Plus
    }

    pub enum LitIntType {
        SignedIntLit(IntTy, Sign),
        UnsignedIntLit(UintTy),
        UnsuffixedIntLit(Sign)
    }

I'd expect that somewhere in the code we'd construct e.g. an SignedIntLit with a Minus in it, but I cannot find a single place that does this after looking for all uses of ast::Minus ast::SignedIntLit, etc.

So my question is, since the sign isn't lexed together with the literal, how is it eventually added to the literal that's stored in in e.g. ast::LitInt?

-- Johan


_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: How are negative int/float literals handled during lexing/parsing?

Kenneth Adam Miller
Hey,

Without answering your question, I think you might be better off posting your question at reddit.com/r/rust because this mailing list has very low participation.

On Wed, Jan 27, 2016 at 11:18 AM, Johan Tibell <[hidden email]> wrote:
Hi!

I'm trying to read the compiler code (mostly because I'm curious) and right now I'm trying to understand the handling of negative integer/float literals.

In token.rs we have:

    pub enum Lit {
        ...
        Integer(ast::Name),
        Float(ast::Name),
        ...
    }

So at this stage these literals are still just (interned) strings. In lexer/mod.rs we can see that the lexer doesn't consume any plus/minus sign, so the above tokens don't contain them.

In ast.rs we have:

    pub enum Expr_ {
        ...
        /// A literal (For example: `1u8`, `"foo"`)
        ExprLit(P<Lit>),
        ...
    }

    pub enum Lit_ {
        ...
        /// An integer literal (`1u8`)
        LitInt(u64, LitIntType),
        /// A float literal (`1f64` or `1E10f64`)
        LitFloat(InternedString, FloatTy),
        /// A float literal without a suffix (`1.0 or 1.0E10`)
        LitFloatUnsuffixed(InternedString),
        ...
    }

    pub enum Sign {
        Minus,
        Plus
    }

    pub enum LitIntType {
        SignedIntLit(IntTy, Sign),
        UnsignedIntLit(UintTy),
        UnsuffixedIntLit(Sign)
    }

I'd expect that somewhere in the code we'd construct e.g. an SignedIntLit with a Minus in it, but I cannot find a single place that does this after looking for all uses of ast::Minus ast::SignedIntLit, etc.

So my question is, since the sign isn't lexed together with the literal, how is it eventually added to the literal that's stored in in e.g. ast::LitInt?

-- Johan


_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev



_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: How are negative int/float literals handled during lexing/parsing?

Starbuck Johnson

If you do make a post please post the link to the mailing list. I would like to know the answer to this question too.


On Wed, Jan 27, 2016, 10:41 AM Kenneth Adam Miller <[hidden email]> wrote:
Hey,

Without answering your question, I think you might be better off posting your question at reddit.com/r/rust because this mailing list has very low participation.

On Wed, Jan 27, 2016 at 11:18 AM, Johan Tibell <[hidden email]> wrote:
Hi!

I'm trying to read the compiler code (mostly because I'm curious) and right now I'm trying to understand the handling of negative integer/float literals.

In token.rs we have:

    pub enum Lit {
        ...
        Integer(ast::Name),
        Float(ast::Name),
        ...
    }

So at this stage these literals are still just (interned) strings. In lexer/mod.rs we can see that the lexer doesn't consume any plus/minus sign, so the above tokens don't contain them.

In ast.rs we have:

    pub enum Expr_ {
        ...
        /// A literal (For example: `1u8`, `"foo"`)
        ExprLit(P<Lit>),
        ...
    }

    pub enum Lit_ {
        ...
        /// An integer literal (`1u8`)
        LitInt(u64, LitIntType),
        /// A float literal (`1f64` or `1E10f64`)
        LitFloat(InternedString, FloatTy),
        /// A float literal without a suffix (`1.0 or 1.0E10`)
        LitFloatUnsuffixed(InternedString),
        ...
    }

    pub enum Sign {
        Minus,
        Plus
    }

    pub enum LitIntType {
        SignedIntLit(IntTy, Sign),
        UnsignedIntLit(UintTy),
        UnsuffixedIntLit(Sign)
    }

I'd expect that somewhere in the code we'd construct e.g. an SignedIntLit with a Minus in it, but I cannot find a single place that does this after looking for all uses of ast::Minus ast::SignedIntLit, etc.

So my question is, since the sign isn't lexed together with the literal, how is it eventually added to the literal that's stored in in e.g. ast::LitInt?

-- Johan


_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev


_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev

_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: How are negative int/float literals handled during lexing/parsing?

Johan Tibell
In reply to this post by Kenneth Adam Miller
I posted it to http://discuss.rust-lang.org/. Thanks!

On Wed, Jan 27, 2016 at 5:41 PM, Kenneth Adam Miller <[hidden email]> wrote:
Hey,

Without answering your question, I think you might be better off posting your question at reddit.com/r/rust because this mailing list has very low participation.

On Wed, Jan 27, 2016 at 11:18 AM, Johan Tibell <[hidden email]> wrote:
Hi!

I'm trying to read the compiler code (mostly because I'm curious) and right now I'm trying to understand the handling of negative integer/float literals.

In token.rs we have:

    pub enum Lit {
        ...
        Integer(ast::Name),
        Float(ast::Name),
        ...
    }

So at this stage these literals are still just (interned) strings. In lexer/mod.rs we can see that the lexer doesn't consume any plus/minus sign, so the above tokens don't contain them.

In ast.rs we have:

    pub enum Expr_ {
        ...
        /// A literal (For example: `1u8`, `"foo"`)
        ExprLit(P<Lit>),
        ...
    }

    pub enum Lit_ {
        ...
        /// An integer literal (`1u8`)
        LitInt(u64, LitIntType),
        /// A float literal (`1f64` or `1E10f64`)
        LitFloat(InternedString, FloatTy),
        /// A float literal without a suffix (`1.0 or 1.0E10`)
        LitFloatUnsuffixed(InternedString),
        ...
    }

    pub enum Sign {
        Minus,
        Plus
    }

    pub enum LitIntType {
        SignedIntLit(IntTy, Sign),
        UnsignedIntLit(UintTy),
        UnsuffixedIntLit(Sign)
    }

I'd expect that somewhere in the code we'd construct e.g. an SignedIntLit with a Minus in it, but I cannot find a single place that does this after looking for all uses of ast::Minus ast::SignedIntLit, etc.

So my question is, since the sign isn't lexed together with the literal, how is it eventually added to the literal that's stored in in e.g. ast::LitInt?

-- Johan


_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev




_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: How are negative int/float literals handled during lexing/parsing?

Johan Tibell

On Wed, Jan 27, 2016 at 5:43 PM, Johan Tibell <[hidden email]> wrote:
I posted it to http://discuss.rust-lang.org/. Thanks!

On Wed, Jan 27, 2016 at 5:41 PM, Kenneth Adam Miller <[hidden email]> wrote:
Hey,

Without answering your question, I think you might be better off posting your question at reddit.com/r/rust because this mailing list has very low participation.

On Wed, Jan 27, 2016 at 11:18 AM, Johan Tibell <[hidden email]> wrote:
Hi!

I'm trying to read the compiler code (mostly because I'm curious) and right now I'm trying to understand the handling of negative integer/float literals.

In token.rs we have:

    pub enum Lit {
        ...
        Integer(ast::Name),
        Float(ast::Name),
        ...
    }

So at this stage these literals are still just (interned) strings. In lexer/mod.rs we can see that the lexer doesn't consume any plus/minus sign, so the above tokens don't contain them.

In ast.rs we have:

    pub enum Expr_ {
        ...
        /// A literal (For example: `1u8`, `"foo"`)
        ExprLit(P<Lit>),
        ...
    }

    pub enum Lit_ {
        ...
        /// An integer literal (`1u8`)
        LitInt(u64, LitIntType),
        /// A float literal (`1f64` or `1E10f64`)
        LitFloat(InternedString, FloatTy),
        /// A float literal without a suffix (`1.0 or 1.0E10`)
        LitFloatUnsuffixed(InternedString),
        ...
    }

    pub enum Sign {
        Minus,
        Plus
    }

    pub enum LitIntType {
        SignedIntLit(IntTy, Sign),
        UnsignedIntLit(UintTy),
        UnsuffixedIntLit(Sign)
    }

I'd expect that somewhere in the code we'd construct e.g. an SignedIntLit with a Minus in it, but I cannot find a single place that does this after looking for all uses of ast::Minus ast::SignedIntLit, etc.

So my question is, since the sign isn't lexed together with the literal, how is it eventually added to the literal that's stored in in e.g. ast::LitInt?

-- Johan


_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev





_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev