PPOTrainer errors with "_restore() takes 3 positional arguments but 4 were given"

[duncanldavis]

Latest ray libraries via pip install on python 3.8

code breaking

trainer = PPOTrainer(config=config)
trainer.restore(checkpoint)

Error
RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=2452, ip=10.139.64.8, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7fc4f840ed60>)
At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: _restore() takes 3 positional arguments but 4 were given
traceback: Traceback (most recent call last):
File "/databricks/python/lib/python3.8/site-packages/ray/serialization.py", line 332, in deserialize_objects
obj = self._deserialize_object(data, metadata, object_ref)
File "/databricks/python/lib/python3.8/site-packages/ray/serialization.py", line 235, in _deserialize_object
return self._deserialize_msgpack_data(data, metadata_fields)
File "/databricks/python/lib/python3.8/site-packages/ray/serialization.py", line 190, in _deserialize_msgpack_data
python_objects = self._deserialize_pickle5_data(pickle5_data)
File "/databricks/python/lib/python3.8/site-packages/ray/serialization.py", line 180, in _deserialize_pickle5_data
obj = pickle.loads(in_band)
TypeError: _restore() takes 3 positional arguments but 4 were given

[duncanldavis]

Ok, it is related to how the ray cluster is setup, when not connecting to the cluster via .init() the trainer works. Working through why everything else works but ppotrainer breaks.

[duncanldavis]

When using num_workers: 0 PPOTrainer works but when it is 1+ I get the attached stack error