Skip to content

[BUG] AsyncEnvPool / ParallelEnv silently drops kwargs during reset() #4520

Open

Description

@coder-jayp

Describe the bug

Currently, asynchronous environments like AsyncEnvPool and ParallelEnv accept **kwargs in their reset() method but silently drop them, never forwarding them to the underlying sub-environments.

To Reproduce

import torch
from torchrl.envs import AsyncEnvPool, GymEnv

def make_env():
    return GymEnv("Pendulum-v1")

if __name__ == "__main__":
    pool = AsyncEnvPool([make_env, make_env])
    
    # We attempt to pass an arbitrary kwarg to the sub-environments.
    # It executes without crashing, but the kwarg is completely dropped 
    # across the worker boundary and never reaches the underlying GymEnvs.
    pool.reset(some_arbitrary_kwarg=True)
    print("Reset finished, but 'some_arbitrary_kwarg' was silently dropped by the pool!")

Expected behavior

The kwargs passed to reset() (such as set_state=True) should be properly forwarded across the worker boundary and executed by the underlying sub-environments.

Screenshots

N/A

Additional context

As discussed in PR #4501, we temporarily added a NotImplementedError to specifically catch set_state=True to prevent it from failing silently, but the underlying architectural bug remains for all kwargs.

Reason and Possible fixes

When the reset() method is called on the pool, it sends a reset command to the workers, but the _run_worker function and the internal multiprocessing communication channels do not currently pack or unpack the kwargs along with the command.

I would like to take up the follow-up PR to implement proper kwarg forwarding across the _run_worker and multiprocessing boundaries so that kwargs are respected by the sub-environments.

Checklist

  • I have checked that there is no similar issue in the repo (required)
  • I have read the documentation (required)
  • I have provided a minimal working example to reproduce the bug (required)

Activity

  1. coder-jayp commented on Oct 3, 2026

    @coder-jayp
    ContributorAuthor

    /assign

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions