Is necessary the manual memory management?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Is necessary the manual memory management?

John Mija
Does make sense to have a language with manual memory management since
it's possible to create stuff of low level with a specialized garbage
collector?
It's good to create drivers, but it's already C.


i.e. Both Native Oberon and Blue Bottle operating systems were
implemented in languages, Oberon and Active Oberon, which have type
safety and concurrent (at least in compiler ulm oberon) GC support.

http://www.inf.ethz.ch/personal/wirth/books/ProjectOberon.pdf


Then, Java is being used in embedded and real-time systems with a
deterministic garbage collection:

http://www.pr.com/press-release/226895

Reply | Threaded
Open this post in threaded view
|

Is necessary the manual memory management?

Patrick Walton
On 10/28/12 4:55 AM, John Mija wrote:
> Does make sense to have a language with manual memory management since
> it's possible to create stuff of low level with a specialized garbage
> collector?
> It's good to create drivers, but it's already C.

Oberon was quite experimental. The Azul C4 collector (the state of the
art, as I understand it) is not open source, would be a tremendous
amount of work to try to replicate, and still has overhead.

For Servo, we need to demonstrate competitive CPU-cycle-for-CPU-cycle
performance against manually memory managed frameworks like Gecko and
WebKit. Relying on global concurrent GC makes the project that much riskier.

Patrick


Reply | Threaded
Open this post in threaded view
|

Is necessary the manual memory management?

Patrick Walton
On 10/28/12 9:03 AM, Bennie Kloosteman wrote:

> AFAIK Azul  is propitiatory  and is not  fast  it makes a huge number of
> small memory allocations and they make changes on linux to make i this
> work rather than on specialized HW .. What is special though is the lack
> of long GC pauses.
>
> That said GCs are getting better the only real issues are
> 1,  GC pauses . Which makes  managing system memory eg for DMA / drivers
>   tricky.
> 2 . Write performance , most GCs have a card table which halves memory
> write performance.
>
> In quite a few cases GC give better performance than malloc ( since
> nursery allocations are just a compare / exchange pointer increment with
> no concern for fragmentation)    and with GCs you can still create and
> manage a static buffer pool if you need performance

For places where it matters (e.g. the binarytrees shootout benchmark) we
have an arena type in the standard library that provides bump allocation.

Making memory pools without special language support isn't particularly
safe; the code could incorrectly return an object to a pool and continue
to hold on to it, creating in effect a dangling pointer. You really want
a region system if you want memory pools to have the same safety
guarantees as ordinary use of the garbage collector.

Patrick


Reply | Threaded
Open this post in threaded view
|

Is necessary the manual memory management?

Sebastian Sylvan
In reply to this post by John Mija
On Sun, Oct 28, 2012 at 4:55 AM, John Mija <jon.mb at proinbox.com> wrote:
> Does make sense to have a language with manual memory management since it's
> possible to create stuff of low level with a specialized garbage collector?

IMO The problem with traditional non-GC languages isn't that it's not
automatic, it's that it's not safe. In other words, the argument for
GC is one for safety, not for "resource management is an unimportant
detail that the system should do".

Safety is the key, not the convenience of not having to think
carefully about resource management. As long as the compiler tells you
when you get it wrong the question is: does it make sense to take one
of the most important performance aspects out of the hands of the
programmer? Automating unnecessary detail is good, but also providing
safe manual control for when you need it is even better.

I think this will be even more the case in the future. Heap sizes are
growing tremendously, and the cost of touching memory is growing too.
GC performance is therefore, in the sense of being relative to the
main CPU workloads, getting worse with time, it seems to me. Putting
the programmer back in the loop here (safely) seems like the right
call.

Seb

--
Sebastian Sylvan

Reply | Threaded
Open this post in threaded view
|

Is necessary the manual memory management?

Niko Matsakis
In reply to this post by John Mija
Regardless of whether manual memory management is desirable as an end
goal, support for it is essentially required if you wish to permit tasks
to exchange ownership of data without copies.  For example, in Servo we
have a double-buffered system where mutable memory buffers are exchanged
between the various parts of the system.  In order to make this safe, we
have to guarantee that this buffer is unaliased at the point where it is
sent---if you know it's unaliased, of course, you also know that you
could safely free it.

As a broader point, it turns out there are a LOT of type system things
you can do if you know something about aliasing (or the lack thereof).
Our current approach to freezing data structures for example is reliant
on this.  Safe array splitting for data parallelism---if we ever go in
that direction---will be reliant on this.  And so forth.  So, supporting
a unique-pointer-like construct makes a lot of sense.


Niko

> John Mija <mailto:jon.mb at proinbox.com>
> October 28, 2012 4:55 AM
> Does make sense to have a language with manual memory management since
> it's possible to create stuff of low level with a specialized garbage
> collector?
> It's good to create drivers, but it's already C.
>
>
> i.e. Both Native Oberon and Blue Bottle operating systems were
> implemented in languages, Oberon and Active Oberon, which have type
> safety and concurrent (at least in compiler ulm oberon) GC support.
>
> http://www.inf.ethz.ch/personal/wirth/books/ProjectOberon.pdf
>
>
> Then, Java is being used in embedded and real-time systems with a
> deterministic garbage collection:
>
> http://www.pr.com/press-release/226895
> _______________________________________________
> Rust-dev mailing list
> Rust-dev at mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20121028/7eb07950/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: compose-unknown-contact.jpg
Type: image/jpeg
Size: 770 bytes
Desc: not available
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20121028/7eb07950/attachment.jpg>

Reply | Threaded
Open this post in threaded view
|

Is necessary the manual memory management?

Matthieu Monrocq
On Sun, Oct 28, 2012 at 8:48 PM, Niko Matsakis <niko at alum.mit.edu> wrote:

> Regardless of whether manual memory management is desirable as an end
> goal, support for it is essentially required if you wish to permit tasks to
> exchange ownership of data without copies.  For example, in Servo we have a
> double-buffered system where mutable memory buffers are exchanged between
> the various parts of the system.  In order to make this safe, we have to
> guarantee that this buffer is unaliased at the point where it is sent?if
> you know it's unaliased, of course, you also know that you could safely
> free it.
>
> As a broader point, it turns out there are a LOT of type system things you
> can do if you know something about aliasing (or the lack thereof). Our
> current approach to freezing data structures for example is reliant on
> this.  Safe array splitting for data parallelism?if we ever go in that
> direction?will be reliant on this.  And so forth.  So, supporting a
> unique-pointer-like construct makes a lot of sense.
>
>
> Niko
>
>
I really think this is the core point: unique/borrowed/shared are less
about memory management than about ownership semantics. It would be
perfectly viable (albeit slower) to treat them all identically in terms of
codegen (ie, GC'em all). On the other hand, ownership semantics provide
both the developer and the compiler with *guarantees* upon which they can
build.

-- Matthieu

>   John Mija <jon.mb at proinbox.com>
>  October 28, 2012 4:55 AM
> Does make sense to have a language with manual memory management since
> it's possible to create stuff of low level with a specialized garbage
> collector?
> It's good to create drivers, but it's already C.
>
>
> i.e. Both Native Oberon and Blue Bottle operating systems were implemented
> in languages, Oberon and Active Oberon, which have type safety and
> concurrent (at least in compiler ulm oberon) GC support.
>
> http://www.inf.ethz.ch/personal/wirth/books/ProjectOberon.pdf
>
>
> Then, Java is being used in embedded and real-time systems with a
> deterministic garbage collection:
>
> http://www.pr.com/press-release/226895
> _______________________________________________
> Rust-dev mailing list
> Rust-dev at mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
>
> _______________________________________________
> Rust-dev mailing list
> Rust-dev at mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20121029/c808ae67/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: compose-unknown-contact.jpg
Type: image/jpeg
Size: 770 bytes
Desc: not available
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20121029/c808ae67/attachment.jpg>

Reply | Threaded
Open this post in threaded view
|

Is necessary the manual memory management?

Lindsey Kuper-2
In reply to this post by Niko Matsakis
On Sun, Oct 28, 2012 at 3:48 PM, Niko Matsakis <niko at alum.mit.edu> wrote:
>
> Regardless of whether manual memory management is desirable as an end
> goal, support for it is essentially required if you wish to permit tasks to
> exchange ownership of data without copies.

Is it fair to say that, in fact, you don't need to do manual memory
management in Rust *unless* you care about efficiently exchanging data
between tasks?

Some people panic at the thought of having to do manual memory
management, but (one might argue) it doesn't feel burdensome in Rust
because the only times you have to use it are times when you'd need to
be thinking low-level anyway -- so, really, it's good that the
language supports reasoning at a low level at the appropriate times.

> As a broader point, it turns out there are a LOT of type system things you
> can do if you know something about aliasing (or the lack thereof). Our
> current approach to freezing data structures for example is reliant on this.
> Safe array splitting for data parallelism?if we ever go in that
> direction?will be reliant on this.  And so forth.  So, supporting a
> unique-pointer-like construct makes a lot of sense.

+1.  Say that, for instance, you want to have a mutable vector that
can be split into two disjoint pieces (so that one task can write to
one piece, and a parallel task to the other), such that an error is
raised if either task touches the other one's piece.  In Haskell, I
think you could enforce this sort of disjoint access entirely at the
type level, using something like the ST monad trick, but then you've
got to layer the parallelism on top of that.  It's possible (I
think!), but it's really complicated.  But!  Rust isn't Haskell, it's
a systems language, and if as Niko is saying, we could accomplish
things like parallel disjoint array access so long as we're willing to
expose a little bit of low-level stuff, then that seems like a strong
argument for exposing that little bit of low-level stuff.  It seems
like a little low-level-ness in Rust will get you a long way.

Lindsey

Reply | Threaded
Open this post in threaded view
|

Is necessary the manual memory management?

Niko Matsakis


Lindsey Kuper wrote:
> Is it fair to say that, in fact, you don't need to do manual memory
> management in Rust*unless*  you care about efficiently exchanging data
> between tasks?

Almost.  pcwalton has been talking about implementing a simple
data-exchange wrapper based around serialization and deserialization.  
This would allow you to just use managed pointers (`@`) everywhere.  The
pieces are all there but not put together into a simple library.


Niko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20121029/a67c4e7c/attachment.html>