Augmented assignment

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Augmented assignment

Chris Morgan
https://github.com/mozilla/rust/issues/5992

(Coming from a Python background I know the concept as augmented
assignment; they are also known as assignment operators. Or just +=,
^=, <<= and the like.)

I want augmented assignment in Rust for all types, so I had a crack at
implementing this. I have an implementation mostly complete in my
augmented-assignment branch
(https://github.com/chris-morgan/rust/compare/augmented-assignment),
but even if I managed to convince the Rust side of rustc that I was
competent, LLVM called my bluff.

The last state of my branch and what appears to be wrong (extra
argument going to the LLVM Call) is documented in #5992.

Is there anyone who would be able to look at my patch and get it to
work (or even figure out just where the wrong is)? I just rebased it
and so it is up to date, but if left for another few weeks it
doubtless will continue to bitrot.
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Masklinn
On 23 août 2013, at 12:44, Chris Morgan <[hidden email]> wrote:

> https://github.com/mozilla/rust/issues/5992
>
> (Coming from a Python background I know the concept as augmented
> assignment; they are also known as assignment operators. Or just +=,
> ^=, <<= and the like.)
>
> I want augmented assignment in Rust for all types, so I had a crack at
> implementing this. I have an implementation mostly complete in my
> augmented-assignment branch
> (https://github.com/chris-morgan/rust/compare/augmented-assignment),
> but even if I managed to convince the Rust side of rustc that I was
> competent, LLVM called my bluff.
>
> The last state of my branch and what appears to be wrong (extra
> argument going to the LLVM Call) is documented in #5992.
>
> Is there anyone who would be able to look at my patch and get it to
> work (or even figure out just where the wrong is)? I just rebased it
> and so it is up to date, but if left for another few weeks it
> doubtless will continue to bitrot.

For the record, I think augmented assignments are a terrible ideas and one of the worst features of python.
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Graydon Hoare
On 13-08-23 11:28 AM, Masklinn wrote:

> For the record, I think augmented assignments are a terrible ideas and one of the worst features of python.

Could you say more (perhaps more constructively)? I believe we have
every intention to support these sorts of overloads longer-term; we
removed the previous support only because it wasn't done terribly well.

-Graydon

_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Matthieu Monrocq



On Fri, Aug 23, 2013 at 8:36 PM, Graydon Hoare <[hidden email]> wrote:
On 13-08-23 11:28 AM, Masklinn wrote:

> For the record, I think augmented assignments are a terrible ideas and one of the worst features of python.

Could you say more (perhaps more constructively)? I believe we have
every intention to support these sorts of overloads longer-term; we
removed the previous support only because it wasn't done terribly well.

-Graydon


Not masklinn, but coming from C++ I have seen operator overloading gone wrong.

The most trivial mistake is to have `+=` and `+` defined so that `a += 5` has a different result than `a = a + 5`; the error can be subtle, and is generally hard to catch because you expect it to be right (about the same issue than `bool operator<(left, right) { return left.a < right.a && left.b < right.b; }`).

If one wants to introduce assignment operators, then, it would be great if this issue was taken care of. It might not be easy though. Especially because of some peculiar issues, such as the fact that `+` can take a constant on either side but `+=` requires a mutable on the left-hand side.

I hope you'll have great ideas.

-- Matthieu
 
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev


_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Artem Egorkine
In reply to this post by Masklinn

Augmented assignments are not unique to python. They are found  in C, Java, Perl, Go, Ruby, JavaScript etc. One could argue that if they were so terrible, they would never have made it to so many programming languages....

On Aug 23, 2013 9:28 PM, "Masklinn" <[hidden email]> wrote:
>
> On 23 août 2013, at 12:44, Chris Morgan <[hidden email]> wrote:
>
> > https://github.com/mozilla/rust/issues/5992
> >
> > (Coming from a Python background I know the concept as augmented
> > assignment; they are also known as assignment operators. Or just +=,
> > ^=, <<= and the like.)
> >
> > I want augmented assignment in Rust for all types, so I had a crack at
> > implementing this. I have an implementation mostly complete in my
> > augmented-assignment branch
> > (https://github.com/chris-morgan/rust/compare/augmented-assignment),
> > but even if I managed to convince the Rust side of rustc that I was
> > competent, LLVM called my bluff.
> >
> > The last state of my branch and what appears to be wrong (extra
> > argument going to the LLVM Call) is documented in #5992.
> >
> > Is there anyone who would be able to look at my patch and get it to
> > work (or even figure out just where the wrong is)? I just rebased it
> > and so it is up to date, but if left for another few weeks it
> > doubtless will continue to bitrot.
>
> For the record, I think augmented assignments are a terrible ideas and one of the worst features of python.
> _______________________________________________
> Rust-dev mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/rust-dev


_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Keegan McAllister
Putting a 'null' value in every type is also a popular language feature, but it's a bad one and Rust deliberately does something different, for safety.  Argument by feature popularity doesn't mean much in a world where most languages are just some new syntax for the same (often broken) concepts.

As for this specific case, I think allowing overloading += makes sense, but I don't feel strongly.

keegan

----- Original Message -----
From: "Artem Egorkine" <[hidden email]>
To: "Masklinn" <[hidden email]>
Cc: [hidden email]
Sent: Friday, August 23, 2013 11:52:49 AM
Subject: Re: [rust-dev] Augmented assignment



Augmented assignments are not unique to python. They are found in C, Java, Perl, Go, Ruby, JavaScript etc. One could argue that if they were so terrible, they would never have made it to so many programming languages....


On Aug 23, 2013 9:28 PM, "Masklinn" < [hidden email] > wrote:

>
> On 23 août 2013, at 12:44, Chris Morgan < [hidden email] > wrote:
>
> > https://github.com/mozilla/rust/issues/5992 
> >
> > (Coming from a Python background I know the concept as augmented
> > assignment; they are also known as assignment operators. Or just +=,
> > ^=, <<= and the like.)
> >
> > I want augmented assignment in Rust for all types, so I had a crack at
> > implementing this. I have an implementation mostly complete in my
> > augmented-assignment branch
> > ( https://github.com/chris-morgan/rust/compare/augmented-assignment ),
> > but even if I managed to convince the Rust side of rustc that I was
> > competent, LLVM called my bluff.
> >
> > The last state of my branch and what appears to be wrong (extra
> > argument going to the LLVM Call) is documented in #5992.
> >
> > Is there anyone who would be able to look at my patch and get it to
> > work (or even figure out just where the wrong is)? I just rebased it
> > and so it is up to date, but if left for another few weeks it
> > doubtless will continue to bitrot.
>
> For the record, I think augmented assignments are a terrible ideas and one of the worst features of python.
> _______________________________________________
> Rust-dev mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/rust-dev 

_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Simon Sapin
In reply to this post by Matthieu Monrocq
Le 23/08/2013 19:47, Matthieu Monrocq a écrit :
> The most trivial mistake is to have `+=` and `+` defined so that `a +=
> 5` has a different result than `a = a + 5`

Would it work to make `a += b` always expand to `a = a + b`, and have
that not be overridable? Or am I missing something else?

--
Simon Sapin
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Graydon Hoare
On 13-08-23 12:16 PM, Simon Sapin wrote:
> Le 23/08/2013 19:47, Matthieu Monrocq a écrit :
>> The most trivial mistake is to have `+=` and `+` defined so that `a +=
>> 5` has a different result than `a = a + 5`
>
> Would it work to make `a += b` always expand to `a = a + b`, and have
> that not be overridable? Or am I missing something else?

Generally these operators want to be able to implement optimized
versions that do not produce and destroy temporaries.

This might be a bit less of an issue when you're working with moved
values, but that just (pardon the pun) moves the implementation
challenge elsewhere: making sure that plus(&a,&b) as you'd have on
non-owned values is the same as plus(a,b) on owned-and-moved values.
It's all a bit subtle.

-Graydon

_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Robin Kruppe
In reply to this post by Graydon Hoare
On Fri, Aug 23, 2013 at 8:36 PM, Graydon Hoare <[hidden email]> wrote:
> On 13-08-23 11:28 AM, Masklinn wrote:
>
>> For the record, I think augmented assignments are a terrible ideas and one of the worst features of python.
>
> Could you say more (perhaps more constructively)? I believe we have
> every intention to support these sorts of overloads longer-term; we
> removed the previous support only because it wasn't done terribly well.

Another Python guy here, I feel similar though I wouldn't word it as strongly.
The problem with += *overloads* is that its sole motivation, a += b
not being simple syntactic sugar for  a = a + b, is also very
confusing.
At least that's the problem in the context of Python. I don't know
enough Rust to decide off-hand whether these points also apply to
Rust, so I'll use Python examples:

Suppose a is a list, b is another list, and c is any non-list iterable
(e.g. a tuple which is basically an immutable list).
One inconsistency is that a + c throws an exception but a += c works
just fine (both work with b in place of c).
Another, more tricky, difference is due to Python's data model where
everything is a reference type and aliasable.
Generally, a = a + b constructs a new and the list a referred to
before isn't affected. In contrast, a += b mutates the list a refers
to.
This is important if the list aliased - e.g., passed a parameter,
stored as a field of an object, or something similar.
At least the latter of these two inconsistencies applies to several
other collection types as well (standard library and third party).

Now, of course this behaviour is intentional and quite useful - it's
the sole reason there is a += overload in the first place (instead of
not defining __iadd__, which means a += b really is equivalent to a =
a + b).
But it's also quite unexpected for most people, and although I've
never been bitten by it personally (I'm try hard not to alias mutable
data), it's not hard to see this causing tricky bugs.
But then again, the only reason it can be observed at all is aliasing
of mutable data. I'm not sure how likely that is in Rust.
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Lindsey Kuper-3
In reply to this post by Masklinn
On Fri, Aug 23, 2013 at 2:28 PM, Masklinn <[hidden email]> wrote:
> For the record, I think augmented assignments are a terrible ideas and one of the worst features of python.

rust-dev is probably not the right place for this sort of
conversation; several points of the code of conduct
(https://github.com/mozilla/rust/wiki/Note-development-policy#conduct)
could apply here.  In any case, the OP is asking a question about a
multi-hundred-line WIP patch on a long-standing bug that has already
been nominated and accepted for a maturity milestone.  Let's respect
the effort they've made, and take it seriously.

Lindsey
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Sebastian Sylvan
In reply to this post by Graydon Hoare


On Fri, Aug 23, 2013 at 12:21 PM, Graydon Hoare <[hidden email]> wrote:
On 13-08-23 12:16 PM, Simon Sapin wrote:
> Le 23/08/2013 19:47, Matthieu Monrocq a écrit :
>> The most trivial mistake is to have `+=` and `+` defined so that `a +=
>> 5` has a different result than `a = a + 5`
>
> Would it work to make `a += b` always expand to `a = a + b`, and have
> that not be overridable? Or am I missing something else?

Generally these operators want to be able to implement optimized
versions that do not produce and destroy temporaries.
 
I sympathize with this point, however I think it should be achieved in by means of something like the GHC RULES pragma[1] where programmers are allowed to tell the compiler about simple optimizations (rewrite rules), in their modules. So if you're doing a math module you could detect a*b+c and change it to a fmadd using a rule, and if there's an earlier pass that expands a+=b to a=a+b, you could then have a rules pragma that says a=a+b rewrites to an in-place add.
 
In practice, rewrite rules have been extraordinarily powerful and pretty much essential in certain cases. E.g. efficient stream processing (stream fusion) relies on it. 

--
Sebastian Sylvan

_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Armin Ronacher
In reply to this post by Graydon Hoare
Hi,

On 23/08/2013 19:36, Graydon Hoare wrote:
> Could you say more (perhaps more constructively)? I believe we have
> every intention to support these sorts of overloads longer-term; we
> removed the previous support only because it wasn't done terribly well.
I think augmented assignments in Python are fine, but they do have some
edge cases.  The most famous one (as this is generally brought up as
Python WTF on various parts of the interwebs) is the one where a mutable
element within an immutable structure is modified:

 >>> a = ([42],)
 >>> a[0] += [23]
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
 >>> a
([42, 23],)

The reason for this is that `a[0] += [23]` expands to
`type(a[0]).__iadd__(a[0], [23])` but afterwards a[0] is overridden with
a[0].

Aside from that I believe += and friends are just fine.  Some people
raised concern that `a = a + [2]` makes a new list but `a += [2]`
modifies a list in place, but I think that is pretty obvious from
looking at the code.


Regards,
Armin
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Flaper87



2013/8/25 Armin Ronacher <[hidden email]>
Hi,


On 23/08/2013 19:36, Graydon Hoare wrote:
Could you say more (perhaps more constructively)? I believe we have
every intention to support these sorts of overloads longer-term; we
removed the previous support only because it wasn't done terribly well.
I think augmented assignments in Python are fine, but they do have some edge cases.  The most famous one (as this is generally brought up as Python WTF on various parts of the interwebs) is the one where a mutable element within an immutable structure is modified:

>>> a = ([42],)
>>> a[0] += [23]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> a
([42, 23],)

The reason for this is that `a[0] += [23]` expands to `type(a[0]).__iadd__(a[0], [23])` but afterwards a[0] is overridden with a[0].

Aside from that I believe += and friends are just fine.  Some people raised concern that `a = a + [2]` makes a new list but `a += [2]` modifies a list in place, but I think that is pretty obvious from looking at the code.


I also think Python's argumented assignments are fine and both a += b vs a + b should behave differently. I personally read a += b as an atomic, thread-safe operation whereas a + b isn't and I beleive that's the way it shoud be. Expanding a += b to a + b doesn't sound right to me.

Anyway, that's my $0.02.

FF

--

_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Flaper87

2013/8/26 Flaper87 <[hidden email]>
argumented

augmented* T_T


--

_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Daniel Micay
In reply to this post by Flaper87
On Mon, Aug 26, 2013 at 5:36 PM, Flaper87 <[hidden email]> wrote:
> I also think Python's argumented assignments are fine and both a += b vs a +
> b should behave differently. I personally read a += b as an atomic,
> thread-safe operation whereas a + b isn't and I beleive that's the way it
> shoud be. Expanding a += b to a + b doesn't sound right to me.
>
> Anyway, that's my $0.02.
>
> FF

In C/C++/Python/Rust, `+=` isn't an atomic operation. Although, in the
safe subset of Rust code there's no way to race on mutable data due to
the inability to share it without locking.
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Flaper87
2013/8/26 Daniel Micay <[hidden email]>
On Mon, Aug 26, 2013 at 5:36 PM, Flaper87 <[hidden email]> wrote:
> I also think Python's argumented assignments are fine and both a += b vs a +
> b should behave differently. I personally read a += b as an atomic,
> thread-safe operation whereas a + b isn't and I beleive that's the way it
> shoud be. Expanding a += b to a + b doesn't sound right to me.
>
> Anyway, that's my $0.02.
>
> FF

In C/C++/Python/Rust, `+=` isn't an atomic operation.

T_T, I knew that, bad wording from my side.
 
Although, in the
safe subset of Rust code there's no way to race on mutable data due to
the inability to share it without locking.

I believe this is good, I wasn't sure though.

FF

--

_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Kevin Ballard-2
In reply to this post by Daniel Micay
On Aug 26, 2013, at 2:46 PM, Daniel Micay <[hidden email]> wrote:

> On Mon, Aug 26, 2013 at 5:36 PM, Flaper87 <[hidden email]> wrote:
>> I also think Python's argumented assignments are fine and both a += b vs a +
>> b should behave differently. I personally read a += b as an atomic,
>> thread-safe operation whereas a + b isn't and I beleive that's the way it
>> shoud be. Expanding a += b to a + b doesn't sound right to me.
>>
>> Anyway, that's my $0.02.
>>
>> FF
>
> In C/C++/Python/Rust, `+=` isn't an atomic operation. Although, in the
> safe subset of Rust code there's no way to race on mutable data due to
> the inability to share it without locking.

My understanding of Python's GIL is that += is indeed atomic because the GIL only locks/unlocks around statements (though I don't believe this is intentional). Though this certainly isn't the case for C/C++.

-Kevin
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Daniel Micay
>On Mon, Aug 26, 2013 at 8:39 PM, Kevin Ballard <[hidden email]> wrote:
> On Aug 26, 2013, at 2:46 PM, Daniel Micay <[hidden email]> wrote:
>
>> On Mon, Aug 26, 2013 at 5:36 PM, Flaper87 <[hidden email]> wrote:
>>> I also think Python's argumented assignments are fine and both a += b vs a +
>>> b should behave differently. I personally read a += b as an atomic,
>>> thread-safe operation whereas a + b isn't and I beleive that's the way it
>>> shoud be. Expanding a += b to a + b doesn't sound right to me.
>>>
>>> Anyway, that's my $0.02.
>>>
>>> FF
>>
>> In C/C++/Python/Rust, `+=` isn't an atomic operation. Although, in the
>> safe subset of Rust code there's no way to race on mutable data due to
>> the inability to share it without locking.
>
> My understanding of Python's GIL is that += is indeed atomic because the GIL only locks/unlocks around statements (though I don't believe this is intentional). Though this certainly isn't the case for C/C++.
>
> -Kevin

The GIL is an implementation detail of CPython, and doesn't exist in
Jython or IronPython since it's not part of the language.
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev
Reply | Threaded
Open this post in threaded view
|

Re: Augmented assignment

Robin Kruppe
In reply to this post by Kevin Ballard-2
On Tue, Aug 27, 2013 at 2:39 AM, Kevin Ballard <[hidden email]> wrote:
[snip]
> My understanding of Python's GIL is that += is indeed atomic because the GIL only locks/unlocks around statements (though I don't believe this is intentional). Though this certainly isn't the case for C/C++.

No, that's incorrect. The GIL locks/unlocks around bytecodes, ans ALSO
in the middle of C functions (built-in or extension module) if whoever
wrote that function thought it might be beneficial. += is compiled
into several bytecode ops: First load what you want to add to, then
call __iadd__ (assuming it is defined) on it, then storing the result
of that (usually the same object) back. The GIL might be released
between any of these steps.

The last step happens because __iadd__ might not be defined (__add__
is the fallback), but even if that disappeared it still wouldn't be
thread-safe.
All this is CPython-specific. AFAIK PyPy tries hard to match this
behavior, but Jython and IronPython probably give even fewer
guarantees. Threading in Python is ugly. In any case, it isn't too
relevant for augmented assignment in Rust, for a broad range of
reasons.
_______________________________________________
Rust-dev mailing list
[hidden email]
https://mail.mozilla.org/listinfo/rust-dev