The Artima Developer Community
Sponsored Link

Weblogs Forum
Other Programmers and Shared-Memory Concurrency

32 replies on 3 pages. Most recent reply: Sep 28, 2007 9:09 PM by Raoul Duke

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 32 replies on 3 pages [ « | 1 2 3 ]
Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Other Programmers and Shared-Memory Concurrency Posted: Sep 20, 2007 8:33 AM
Reply to this message Reply
Advertisement
> My preferred tool to work with is the "nanothreads"
> module of Simon Wittber's FibraNet
> (http://pypi.python.org/pypi/FibraNet/) package. It
> provides a nice threadlet API using generators. You can
> optionally yield UNBLOCK from a nanothread and the next
> iteration will be scheduled in a separate thread. This
> works nice when you know you are about to perform some
> blocking IO or similar.

I'd prefer to upgrade to proper Python 2.5 coroutines and implement full Stackless tasklet functionality. What I'd also like to see is a minimal API or a protocol specification a la WSGI for generators. So any generator that provides this API or gets decorated appropriately can be properly scheduled and become an "actor" or "nanothread". The Python community shall be contentious about "too many frameworks".

Note that I have resolved one big stumbling block for transparent distribution of CPython generators across OS level processes, namely that they were not copyable / pickable.

See also the cookbook recipes:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/528949
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/528951

I always wondered why Stackless Python was not better integrated with the processing package since it provided pickable tasklets for quite a while.

>
> I also like and use the "processing"
> (http://www.python.org/pypi/processing) package to manage
> splitting some tasks off into separate processes. I like
> its API based on the threading module.
>
> For me, I would like to see these two approaches
> integrated so that I could use the nanothreads approach
> but perhaps yield PROCESS to have the scheduler spawn off
> a processing module process.

I think that's a bad idea. A generator shouldn't affect the behaviour of the scheduler deterministically and enforce a thread or process to be spawned on it's own behalf. This violates encapsulation. Instead distribution shall be layered and balanced by the generator framework. A generator that spawns its own process might produce values that no other actor currently needs.

Think about a simple generator:


def f():
i = 0
while True:
yield (PROCESS, i)


which triggers a new process together with each value it yields.

Instead a generator shall only tell what it provides and needs and how it might be continued. In a sense the latter is obvious because there are two forms of yielding values:

yield a # no send required. generator can produce new
# value immediately

b = yield a # send required. generator is blocking and
# expects new value -- equivalent to require
# clause in Erlang

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Other Programmers and Shared-Memory Concurrency Posted: Sep 20, 2007 8:35 AM
Reply to this message Reply
> > I haven't read the whole thread from the start, but why
> > not use the Actor model fully? i.e. every object in the
> > language should be a different thread, and invocation of
> a
> > method should be a computational job posted to the job
> > queue of an object's thread. Condition variables should
> be
> > automatically inserted when waiting for a result to be
> > available.
>
> Can we share data between threads in the Actors model? I
> think the model is not designed for that, but I'm not
> sure. Anyway, it looks like this would be a similar
> approach taken in Erlang with message-passing concurrency,
> but I don't think it would be a good idea to prefer one
> concurrency model over others for a general-purpose
> language.

Certainly. Data can be freely shared in the Actor model, because each actor is a separate thread.

Raoul Duke

Posts: 127
Nickname: raoulduke
Registered: Apr, 2006

Re: Other Programmers and Shared-Memory Concurrency Posted: Sep 28, 2007 9:09 PM
Reply to this message Reply
> Honestly I'd prefer alternatives using dataflow variables
> and implicit synchronization or declarative programming
> idioms where the solver engine cares for distribution and
> not the application programmer.

CTM, yes?

http://lambda-the-ultimate.org/node/2457#comment-37055

Flat View: This topic has 32 replies on 3 pages [ « | 1  2  3 ]
Topic: Other Programmers and Shared-Memory Concurrency Previous Topic   Next Topic Topic: Thinking in Java 4e Solution Guide Now Available


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2017 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us