The mechanism that's currently documented for passing remoteinfo to autopara ops in docs/source/overview.queued.md is quite complex (in its most general form). Even though I came up with it, I've mostly stopped using it.
Instead, I've switched to explicitly passing RemoteInfo objects (or dicts) in the actual function call, but obtaining those by importing from another file, to separate the task and the details for parallelizing on any given machine. E.g. I have a python file called remote_details.py, which contains
MD_operation = {"one_hpc_machine": { dict with remoteinfo stuff for that machine },
"other_hpc_machine": {dict with remoteinfo stuff fot this other machine}
}
which I then import, and when I call the autopara MD function I pass remoteinfo = remote_details.MD_operation["one_hpc_machine"].
I think this is versatile enough, and simpler from the view of the wfl implementation. We should consider removing the env-var based code, and revising the docs as needed.
The mechanism that's currently documented for passing remoteinfo to autopara ops in
docs/source/overview.queued.mdis quite complex (in its most general form). Even though I came up with it, I've mostly stopped using it.Instead, I've switched to explicitly passing
RemoteInfoobjects (or dicts) in the actual function call, but obtaining those by importing from another file, to separate the task and the details for parallelizing on any given machine. E.g. I have a python file calledremote_details.py, which containswhich I then import, and when I call the autopara MD function I pass
remoteinfo = remote_details.MD_operation["one_hpc_machine"].I think this is versatile enough, and simpler from the view of the
wflimplementation. We should consider removing the env-var based code, and revising the docs as needed.