Weblogs Forum - Other Programmers and Shared-Memory Concurrency

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
Other Programmers and Shared-Memory Concurrency

32 replies on 3 pages. Most recent reply: Sep 28, 2007 6:09 PM by Raoul Duke

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 32 replies on 3 pages [ « | 1 2 3 ]

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Other Programmers and Shared-Memory Concurrency

Posted: Sep 20, 2007 5:33 AM

> My preferred tool to work with is the "nanothreads"
> module of Simon Wittber's FibraNet
> (http://pypi.python.org/pypi/FibraNet/) package. It
> provides a nice threadlet API using generators. You can
> optionally yield UNBLOCK from a nanothread and the next
> iteration will be scheduled in a separate thread. This
> works nice when you know you are about to perform some
> blocking IO or similar.

I'd prefer to upgrade to proper Python 2.5 coroutines and implement full Stackless tasklet functionality. What I'd also like to see is a minimal API or a protocol specification a la WSGI for generators. So any generator that provides this API or gets decorated appropriately can be properly scheduled and become an "actor" or "nanothread". The Python community shall be contentious about "too many frameworks".

Note that I have resolved one big stumbling block for transparent distribution of CPython generators across OS level processes, namely that they were not copyable / pickable.

See also the cookbook recipes:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/528949
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/528951

I always wondered why Stackless Python was not better integrated with the processing package since it provided pickable tasklets for quite a while.

>
> I also like and use the "processing"
> (http://www.python.org/pypi/processing) package to manage
> splitting some tasks off into separate processes. I like
> its API based on the threading module.
>
> For me, I would like to see these two approaches
> integrated so that I could use the nanothreads approach
> but perhaps yield PROCESS to have the scheduler spawn off
> a processing module process.

I think that's a bad idea. A generator shouldn't affect the behaviour of the scheduler deterministically and enforce a thread or process to be spawned on it's own behalf. This violates encapsulation. Instead distribution shall be layered and balanced by the generator framework. A generator that spawns its own process might produce values that no other actor currently needs.

Think about a simple generator:


def f():
    i = 0
    while True:
        yield (PROCESS, i)

which triggers a new process together with each value it yields.

Instead a generator shall only tell what it provides and needs and how it might be continued. In a sense the latter is obvious because there are two forms of yielding values:


yield a  # no send required. generator can produce new
         # value immediately

b = yield a  # send required. generator is blocking and 
             # expects new value -- equivalent to require
             # clause in Erlang

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Other Programmers and Shared-Memory Concurrency

Posted: Sep 20, 2007 5:35 AM

> > I haven't read the whole thread from the start, but why
> > not use the Actor model fully? i.e. every object in the
> > language should be a different thread, and invocation of
> a
> > method should be a computational job posted to the job
> > queue of an object's thread. Condition variables should
> be
> > automatically inserted when waiting for a result to be
> > available.
>
> Can we share data between threads in the Actors model? I
> think the model is not designed for that, but I'm not
> sure. Anyway, it looks like this would be a similar
> approach taken in Erlang with message-passing concurrency,
> but I don't think it would be a good idea to prefer one
> concurrency model over others for a general-purpose
> language.

Certainly. Data can be freely shared in the Actor model, because each actor is a separate thread.

Raoul Duke

Posts: 127
Nickname: raoulduke
Registered: Apr, 2006

Re: Other Programmers and Shared-Memory Concurrency

Posted: Sep 28, 2007 6:09 PM

> Honestly I'd prefer alternatives using dataflow variables
> and implicit synchronization or declarative programming
> idioms where the solver engine cares for distribution and
> not the application programmer.

CTM, yes?

http://lambda-the-ultimate.org/node/2457#comment-37055

Flat View: This topic has 32 replies on 3 pages [ « | 1 2 3 ]

Previous Topic

Next Topic


	Web Artima.com