Expose asyncio to python3 rplugins, removing python3.3 support

[bfredl]

Currently, handling events beyond nvim itself is very limited, the only way additional sources can be added is with separate threads invoking nvim.async_call. A problem is that the implemenation is dynamic between two event loops: asyncio and pyuv, so plugins can't use them.

The plan is to require asyncio on python3 so we can expose asyncio event loop to python3 rplugins. Python3.3 is hereby unsupported. Python2.7 rplugins is considered legacy: existing py2.7 rplugins remain supported for now, but new features might not be available (python2 via vim :python interface remain supported of course)

I know issues on windows has held up this change in the past (#243). We might need to change the asyncio backend on windows. IIRC there is even a libuv based backend, but I don't know it's status. I will look into this tomorrow.

I will make example and test this with subprocess communication, using msgpack to comminucate with a python helper process. I know deoplete wants to use the feature for multiprocess. Ping @Shougo

[Shougo]

Thanks. I want to use the feature for deoplete.

[Shougo]

I will make example and test this with subprocess communication, using msgpack to comminucate with a python helper process. I know deoplete wants to use the feature for multiprocess. Ping @Shougo

@bfredl Ping.

[bfredl]

Sorry for the delay, this works so far:

import neovim
import asyncio
import sys
from functools import partial

class MySubprocess(asyncio.SubprocessProtocol):
    def __init__(self, plugin):
        self.plugin = plugin
        self.nvim = plugin.nvim

    def connection_made(self, transport):
        self.nvim.async_call(self.plugin._on_connection, transport)
        self._transport = transport
        if isinstance(transport, asyncio.SubprocessTransport):
            self._transport = transport.get_pipe_transport(0)

    def pipe_data_received(self, fd, data):
        self.nvim.async_call(self.plugin._on_output, fd, data)

    def process_exited(self):
        pass

@neovim.plugin
class TestPlugin(object):

    def __init__(self, nvim):
        self.nvim = nvim

    @neovim.function("SubTest", sync=True)
    def subtest(self, args):
        code = "print('test')"
        create = self.nvim.loop.subprocess_exec(partial(MySubprocess,self),
                                  sys.executable, '-c', code,
                                  stdin=None, stderr=None)
        hnd = self.nvim.loop.create_task(create)

    def _on_connection(self, transport):
        self.nvim.out_write("connected!\n")
        self.sub_stdin = transport.get_pipe_transport(0)
        #self.sub_stdin.write(b"spam")

    def _on_output(self, fd, data):
        self.nvim.out_write(repr(fd)+" "+repr(data)+"\n")

But it is not very convenient. We probably want nvim.subprocess(out_cb,err_cb) wrapper or at least result = nvim.yieldfrom(coro) so that coroutines can be used inside greenlet handlers (i e method calls from nvim)

[Shougo]

Thank you!

I will check it.

[Shougo]

It is conflicting.
@bfredl Can you fix the conflict?

[Shougo]

@bfredl Ping.

[bfredl]

Sorry for the delay. I will add some documentation also tomorrow maybe, then we can merge this. Wrappers can be a follow-up PR. I e things like #72 can be implement much simpler, no need to traverse layers, can just implement thin wrapper directly in nvim module.

[Shougo]

Thanks. I will add support the feature in deoplete later.

[Shougo]

@bfredl I want to write input to sub process in asyncio loop.
It is possible?

[bfredl]

@Shougo Yes, in the example above, in _on_commection, call transport.get_pipe_transport(0) to get the stdin pipe. On this object you can call .write(data).

[Shougo]

Thank you.

        create = self.nvim.loop.subprocess_exec(partial(MySubprocess,self),
                                  sys.executable, '-c', code,
                                  stdin=None, stderr=None)

I think if stdin is None, the input feature will be disabled.

[Shougo]

@bfredl Ping.

[Shougo]

And I think the feature is not supported in Windows.

[Shougo]

#294 (comment)

The test is works for me.

Hm.
I will look into it later.

[Shougo]

@bfredl No problem. deoplete asyncio works for me.