比较提交
2 次代码提交
main
...
async_quer
| 作者 | SHA1 | 提交日期 | |
|---|---|---|---|
| 0aba9e4f67 | |||
| a0905ad3b5 |
27
README.md
27
README.md
@ -1,11 +1,13 @@
|
|||||||
# Imitation Learning algorithms and Co-training for Mobile ALOHA
|
# ACT: Action Chunking with Transformers
|
||||||
|
|
||||||
|
### *New*: [ACT tuning tips](https://docs.google.com/document/d/1FVIZfoALXg_ZkYKaYVh-qOlaXveq5CtvJHXkY25eYhs/edit?usp=sharing)
|
||||||
|
TL;DR: if your ACT policy is jerky or pauses in the middle of an episode, just train for longer! Success rate and smoothness can improve way after loss plateaus.
|
||||||
|
|
||||||
#### Project Website: https://mobile-aloha.github.io/
|
#### Project Website: https://tonyzhaozh.github.io/aloha/
|
||||||
|
|
||||||
This repo contains the implementation of ACT, Diffusion Policy and VINN, together with 2 simulated environments:
|
This repo contains the implementation of ACT, together with 2 simulated environments:
|
||||||
Transfer Cube and Bimanual Insertion. You can train and evaluate them in sim or real.
|
Transfer Cube and Bimanual Insertion. You can train and evaluate ACT in sim or real.
|
||||||
For real, you would also need to install [Mobile ALOHA](https://github.com/MarkFzp/mobile-aloha). This repo is forked from the [ACT repo](https://github.com/tonyzhaozh/act).
|
For real, you would also need to install [ALOHA](https://github.com/tonyzhaozh/aloha).
|
||||||
|
|
||||||
### Updates:
|
### Updates:
|
||||||
You can find all scripted/human demo for simulated environments [here](https://drive.google.com/drive/folders/1gPR03v05S1xiInoVJn7G7VJ9pDCnxq9O?usp=share_link).
|
You can find all scripted/human demo for simulated environments [here](https://drive.google.com/drive/folders/1gPR03v05S1xiInoVJn7G7VJ9pDCnxq9O?usp=share_link).
|
||||||
@ -33,18 +35,17 @@ You can find all scripted/human demo for simulated environments [here](https://d
|
|||||||
pip install pyyaml
|
pip install pyyaml
|
||||||
pip install rospkg
|
pip install rospkg
|
||||||
pip install pexpect
|
pip install pexpect
|
||||||
pip install mujoco==2.3.7
|
pip install mujoco
|
||||||
pip install dm_control==1.0.14
|
pip install dm_control
|
||||||
pip install opencv-python
|
pip install opencv-python
|
||||||
pip install matplotlib
|
pip install matplotlib
|
||||||
pip install einops
|
pip install einops
|
||||||
pip install packaging
|
pip install packaging
|
||||||
pip install h5py
|
pip install h5py
|
||||||
pip install ipython
|
pip install ipython
|
||||||
|
pip install ray==2.7.1
|
||||||
cd act/detr && pip install -e .
|
cd act/detr && pip install -e .
|
||||||
|
|
||||||
- also need to install https://github.com/ARISE-Initiative/robomimic/tree/r2d2 (note the r2d2 branch) for Diffusion Policy by `pip install -e .`
|
|
||||||
|
|
||||||
### Example Usages
|
### Example Usages
|
||||||
|
|
||||||
To set up a new terminal, run:
|
To set up a new terminal, run:
|
||||||
@ -52,7 +53,7 @@ To set up a new terminal, run:
|
|||||||
conda activate aloha
|
conda activate aloha
|
||||||
cd <path to act repo>
|
cd <path to act repo>
|
||||||
|
|
||||||
### Simulated experiments (LEGACY table-top ALOHA environments)
|
### Simulated experiments
|
||||||
|
|
||||||
We use ``sim_transfer_cube_scripted`` task in the examples below. Another option is ``sim_insertion_scripted``.
|
We use ``sim_transfer_cube_scripted`` task in the examples below. Another option is ``sim_insertion_scripted``.
|
||||||
To generated 50 episodes of scripted data, run:
|
To generated 50 episodes of scripted data, run:
|
||||||
@ -60,12 +61,10 @@ To generated 50 episodes of scripted data, run:
|
|||||||
python3 record_sim_episodes.py --task_name sim_transfer_cube_scripted --dataset_dir <data save dir> --num_episodes 50
|
python3 record_sim_episodes.py --task_name sim_transfer_cube_scripted --dataset_dir <data save dir> --num_episodes 50
|
||||||
|
|
||||||
To can add the flag ``--onscreen_render`` to see real-time rendering.
|
To can add the flag ``--onscreen_render`` to see real-time rendering.
|
||||||
To visualize the simulated episodes after it is collected, run
|
To visualize the episode after it is collected, run
|
||||||
|
|
||||||
python3 visualize_episodes.py --dataset_dir <data save dir> --episode_idx 0
|
python3 visualize_episodes.py --dataset_dir <data save dir> --episode_idx 0
|
||||||
|
|
||||||
Note: to visualize data from the mobile-aloha hardware, use the visualize_episodes.py from https://github.com/MarkFzp/mobile-aloha
|
|
||||||
|
|
||||||
To train ACT:
|
To train ACT:
|
||||||
|
|
||||||
# Transfer Cube task
|
# Transfer Cube task
|
||||||
@ -81,5 +80,3 @@ You can also add ``--onscreen_render`` to see real-time rendering during evaluat
|
|||||||
For real-world data where things can be harder to model, train for at least 5000 epochs or 3-4 times the length after the loss has plateaued.
|
For real-world data where things can be harder to model, train for at least 5000 epochs or 3-4 times the length after the loss has plateaued.
|
||||||
Please refer to [tuning tips](https://docs.google.com/document/d/1FVIZfoALXg_ZkYKaYVh-qOlaXveq5CtvJHXkY25eYhs/edit?usp=sharing) for more info.
|
Please refer to [tuning tips](https://docs.google.com/document/d/1FVIZfoALXg_ZkYKaYVh-qOlaXveq5CtvJHXkY25eYhs/edit?usp=sharing) for more info.
|
||||||
|
|
||||||
### [ACT tuning tips](https://docs.google.com/document/d/1FVIZfoALXg_ZkYKaYVh-qOlaXveq5CtvJHXkY25eYhs/edit?usp=sharing)
|
|
||||||
TL;DR: if your ACT policy is jerky or pauses in the middle of an episode, just train for longer! Success rate and smoothness can improve way after loss plateaus.
|
|
||||||
|
|||||||
146
commands.txt
146
commands.txt
@ -205,56 +205,7 @@ CUDA_VISIBLE_DEVICES=1 python3 imitate_episodes.py \
|
|||||||
--batch_size 32 --lr 1e-4 --seed 0 \
|
--batch_size 32 --lr 1e-4 --seed 0 \
|
||||||
--num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000
|
--num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000
|
||||||
|
|
||||||
# try chunk size 64, no cotrain
|
|
||||||
|
|
||||||
conda activate mobile
|
|
||||||
export MUJOCO_GL=egl
|
|
||||||
cd /home/tonyzhao/Research/act-plus-plus
|
|
||||||
CUDA_VISIBLE_DEVICES=0 python3 imitate_episodes.py \
|
|
||||||
--task_name aloha_mobile_wipe_wine \
|
|
||||||
--ckpt_dir /scr/tonyzhao/train_logs/wipe_wine_diffusion_augmentation_chunk64_seed0 \
|
|
||||||
--policy_class Diffusion --chunk_size 64 \
|
|
||||||
--batch_size 32 --lr 1e-4 --seed 0 \
|
|
||||||
--num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000
|
|
||||||
|
|
||||||
# chunk 64 with cotrain
|
|
||||||
|
|
||||||
conda activate mobile
|
|
||||||
export MUJOCO_GL=egl
|
|
||||||
cd /home/tonyzhao/Research/act-plus-plus
|
|
||||||
CUDA_VISIBLE_DEVICES=1 python3 imitate_episodes.py \
|
|
||||||
--task_name aloha_mobile_wipe_wine_cotrain \
|
|
||||||
--ckpt_dir /scr/tonyzhao/train_logs/wipe_wine_cotrain_diffusion_augmentation_chunk64_seed0 \
|
|
||||||
--policy_class Diffusion --chunk_size 64 \
|
|
||||||
--batch_size 32 --lr 1e-4 --seed 0 \
|
|
||||||
--num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# chunk 64 with cotrain + EMA
|
|
||||||
|
|
||||||
conda activate mobile
|
|
||||||
export MUJOCO_GL=egl
|
|
||||||
cd /home/tonyzhao/Research/act-plus-plus
|
|
||||||
CUDA_VISIBLE_DEVICES=0 python3 imitate_episodes.py \
|
|
||||||
--task_name aloha_mobile_wipe_wine_2_cotrain \
|
|
||||||
--ckpt_dir /scr/tonyzhao/train_logs/wipe_wine_cotrain_diffusion_augmentation_chunk64_ema_seed0 \
|
|
||||||
--policy_class Diffusion --chunk_size 64 \
|
|
||||||
--batch_size 32 --lr 1e-4 --seed 0 \
|
|
||||||
--num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000
|
|
||||||
|
|
||||||
|
|
||||||
# chunk 64 with cotrain + EMA + 3e-4
|
|
||||||
|
|
||||||
conda activate mobile
|
|
||||||
export MUJOCO_GL=egl
|
|
||||||
cd /home/tonyzhao/Research/act-plus-plus
|
|
||||||
CUDA_VISIBLE_DEVICES=1 python3 imitate_episodes.py \
|
|
||||||
--task_name aloha_mobile_wipe_wine_2_cotrain \
|
|
||||||
--ckpt_dir /scr/tonyzhao/train_logs/wipe_wine_cotrain_diffusion_augmentation_chunk64_ema_3e-4_seed0 \
|
|
||||||
--policy_class Diffusion --chunk_size 64 \
|
|
||||||
--batch_size 32 --lr 3e-4 --seed 0 \
|
|
||||||
--num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000
|
|
||||||
|
|
||||||
|
|
||||||
######################## VINN ########################
|
######################## VINN ########################
|
||||||
@ -400,6 +351,9 @@ python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/
|
|||||||
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME}
|
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
conda activate mobile
|
conda activate mobile
|
||||||
export CUDA_VISIBLE_DEVICES=1
|
export CUDA_VISIBLE_DEVICES=1
|
||||||
cd /home/tonyzhao/Research/act-plus-plus
|
cd /home/tonyzhao/Research/act-plus-plus
|
||||||
@ -418,83 +372,6 @@ python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
# push chair task
|
|
||||||
|
|
||||||
conda activate mobile
|
|
||||||
export CUDA_VISIBLE_DEVICES=0
|
|
||||||
cd /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning
|
|
||||||
python3 train.py --task aloha_mobile_chair_truncated --cam_name cam_high --seed 0
|
|
||||||
python3 train.py --task aloha_mobile_chair_truncated --cam_name cam_left_wrist --seed 0
|
|
||||||
python3 train.py --task aloha_mobile_chair_truncated --cam_name cam_right_wrist --seed 0
|
|
||||||
|
|
||||||
cd /home/tonyzhao/Research/act-plus-plus
|
|
||||||
TASK_NAME=aloha_mobile_chair_truncated
|
|
||||||
DATA_NAME=aloha_mobile_chair_truncated
|
|
||||||
python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
|
|
||||||
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME}
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
conda activate mobile
|
|
||||||
export CUDA_VISIBLE_DEVICES=1
|
|
||||||
cd /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning
|
|
||||||
python3 train.py --task aloha_mobile_chair_truncated_cotrain --cam_name cam_high --seed 0
|
|
||||||
python3 train.py --task aloha_mobile_chair_truncated_cotrain --cam_name cam_left_wrist --seed 0
|
|
||||||
python3 train.py --task aloha_mobile_chair_truncated_cotrain --cam_name cam_right_wrist --seed 0
|
|
||||||
|
|
||||||
cd /home/tonyzhao/Research/act-plus-plus
|
|
||||||
TASK_NAME=aloha_mobile_chair_truncated_cotrain
|
|
||||||
DATA_NAME=aloha_mobile_chair_truncated
|
|
||||||
python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
|
|
||||||
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME}
|
|
||||||
|
|
||||||
|
|
||||||
# cache feature again for wipe wine
|
|
||||||
|
|
||||||
conda activate mobile
|
|
||||||
export CUDA_VISIBLE_DEVICES=0
|
|
||||||
cd /home/tonyzhao/Research/act-plus-plus
|
|
||||||
TASK_NAME=aloha_mobile_wipe_wine
|
|
||||||
DATA_NAME=aloha_mobile_wipe_wine
|
|
||||||
python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
|
|
||||||
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME}
|
|
||||||
|
|
||||||
cd /home/tonyzhao/Research/act-plus-plus
|
|
||||||
TASK_NAME=aloha_mobile_wipe_wine_cotrain
|
|
||||||
DATA_NAME=aloha_mobile_wipe_wine
|
|
||||||
python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
|
|
||||||
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME}
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# run on real robot
|
|
||||||
|
|
||||||
TASK_NAME=aloha_mobile_chair_truncated
|
|
||||||
python3 vinn_select_k.py \
|
|
||||||
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME} \
|
|
||||||
--ckpt_dir /scr/tonyzhao/train_logs/VINN-eval-${TASK_NAME}-seed-0
|
|
||||||
|
|
||||||
python3 vinn_eval.py \
|
|
||||||
--dataset_dir /scr/tonyzhao/datasets/sim_transfer_cube_scripted \
|
|
||||||
--model_dir /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
|
|
||||||
--ckpt_dir /scr/tonyzhao/train_logs/VINN-eval-${TASK_NAME}-seed-0 \
|
|
||||||
--task_name $TASK_NAME
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
TASK_NAME=aloha_mobile_chair_truncated
|
|
||||||
python3 vinn_select_k.py \
|
|
||||||
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME} \
|
|
||||||
--ckpt_dir /scr/tonyzhao/train_logs/VINN-eval-${TASK_NAME}-seed-0
|
|
||||||
|
|
||||||
python3 vinn_eval.py \
|
|
||||||
--dataset_dir /scr/tonyzhao/datasets/sim_transfer_cube_scripted \
|
|
||||||
--model_dir /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
|
|
||||||
--ckpt_dir /scr/tonyzhao/train_logs/VINN-eval-${TASK_NAME}-seed-0 \
|
|
||||||
--task_name $TASK_NAME
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# eval on real robot
|
# eval on real robot
|
||||||
|
|
||||||
conda activate aloha
|
conda activate aloha
|
||||||
@ -517,6 +394,23 @@ python3 vinn_eval.py \
|
|||||||
--task_name $TASK_NAME
|
--task_name $TASK_NAME
|
||||||
|
|
||||||
|
|
||||||
|
# debug async model query
|
||||||
|
|
||||||
|
|
||||||
|
conda activate mobile
|
||||||
|
export MUJOCO_GL=egl
|
||||||
|
cd /home/tonyzhao/Research/act-plus-plus
|
||||||
|
CUDA_VISIBLE_DEVICES=0 python3 imitate_episodes.py \
|
||||||
|
--task_name sim_transfer_cube_scripted \
|
||||||
|
--ckpt_dir /scr/tonyzhao/train_logs/cube_scripted_diffusion_sweep_6_noEMA_seed1 \
|
||||||
|
--policy_class Diffusion --chunk_size 32 \
|
||||||
|
--batch_size 32 --lr 1e-4 --seed 1 \
|
||||||
|
--num_steps 200000 --eval_every 6000 --validate_every 6000 --save_every 6000 --eval
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
---------------------------------------------------------------------------------------
|
---------------------------------------------------------------------------------------
|
||||||
|
|
||||||
NOTE: chunk size cannot be any number, try before launching
|
NOTE: chunk size cannot be any number, try before launching
|
||||||
|
|||||||
@ -10,6 +10,8 @@ from tqdm import tqdm
|
|||||||
from einops import rearrange
|
from einops import rearrange
|
||||||
import wandb
|
import wandb
|
||||||
import time
|
import time
|
||||||
|
import ray
|
||||||
|
from ray.util.queue import Queue
|
||||||
from torchvision import transforms
|
from torchvision import transforms
|
||||||
|
|
||||||
from constants import FPS
|
from constants import FPS
|
||||||
@ -27,6 +29,8 @@ from sim_env import BOX_POSE
|
|||||||
import IPython
|
import IPython
|
||||||
e = IPython.embed
|
e = IPython.embed
|
||||||
|
|
||||||
|
ray.init()
|
||||||
|
|
||||||
def get_auto_index(dataset_dir):
|
def get_auto_index(dataset_dir):
|
||||||
max_idx = 1000
|
max_idx = 1000
|
||||||
for i in range(max_idx+1):
|
for i in range(max_idx+1):
|
||||||
@ -209,7 +213,7 @@ def get_image(ts, camera_names, rand_crop_resize=False):
|
|||||||
curr_image = rearrange(ts.observation['images'][cam_name], 'h w c -> c h w')
|
curr_image = rearrange(ts.observation['images'][cam_name], 'h w c -> c h w')
|
||||||
curr_images.append(curr_image)
|
curr_images.append(curr_image)
|
||||||
curr_image = np.stack(curr_images, axis=0)
|
curr_image = np.stack(curr_images, axis=0)
|
||||||
curr_image = torch.from_numpy(curr_image / 255.0).float().cuda().unsqueeze(0)
|
curr_image = torch.from_numpy(curr_image / 255.0).float().unsqueeze(0)
|
||||||
|
|
||||||
if rand_crop_resize:
|
if rand_crop_resize:
|
||||||
print('rand crop resize is used!')
|
print('rand crop resize is used!')
|
||||||
@ -225,6 +229,22 @@ def get_image(ts, camera_names, rand_crop_resize=False):
|
|||||||
return curr_image
|
return curr_image
|
||||||
|
|
||||||
|
|
||||||
|
@ray.remote(num_gpus=0.1)
|
||||||
|
class PolicyLoop:
|
||||||
|
def __init__(self, policy, input_queue, action_queue):
|
||||||
|
self.input_queue = input_queue
|
||||||
|
self.action_queue = action_queue
|
||||||
|
self.policy = policy # Replace with your AI model initialization
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
while True:
|
||||||
|
input = self.input_queue.get()
|
||||||
|
obs = input['obs'].cuda()
|
||||||
|
qpos = input['qpos'].cuda()
|
||||||
|
action = self.policy(qpos, obs)
|
||||||
|
self.action_queue.put({'action': action.cpu(), 'query_wall_t': input['query_wall_t'], 'query_t': input['query_t']})
|
||||||
|
|
||||||
|
|
||||||
def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
|
def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
|
||||||
set_seed(1000)
|
set_seed(1000)
|
||||||
ckpt_dir = config['ckpt_dir']
|
ckpt_dir = config['ckpt_dir']
|
||||||
@ -242,6 +262,8 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
|
|||||||
actuator_config = config['actuator_config']
|
actuator_config = config['actuator_config']
|
||||||
use_actuator_net = actuator_config['actuator_network_dir'] is not None
|
use_actuator_net = actuator_config['actuator_network_dir'] is not None
|
||||||
|
|
||||||
|
async_query = True
|
||||||
|
|
||||||
# load policy and stats
|
# load policy and stats
|
||||||
ckpt_path = os.path.join(ckpt_dir, ckpt_name)
|
ckpt_path = os.path.join(ckpt_dir, ckpt_name)
|
||||||
policy = make_policy(policy_class, policy_config)
|
policy = make_policy(policy_class, policy_config)
|
||||||
@ -293,6 +315,17 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
|
|||||||
else:
|
else:
|
||||||
post_process = lambda a: a * stats['action_std'] + stats['action_mean']
|
post_process = lambda a: a * stats['action_std'] + stats['action_mean']
|
||||||
|
|
||||||
|
# async
|
||||||
|
if async_query:
|
||||||
|
ray.put(policy)
|
||||||
|
# Create the communication queues
|
||||||
|
input_queue = Queue()
|
||||||
|
action_queue = Queue()
|
||||||
|
# Start the robot AI loop actor
|
||||||
|
for _ in range(2):
|
||||||
|
policy_loop = PolicyLoop.remote(policy, input_queue, action_queue)
|
||||||
|
policy_loop.run.remote()
|
||||||
|
|
||||||
# load environment
|
# load environment
|
||||||
if real_robot:
|
if real_robot:
|
||||||
from aloha_scripts.robot_utils import move_grippers # requires aloha
|
from aloha_scripts.robot_utils import move_grippers # requires aloha
|
||||||
@ -304,7 +337,7 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
|
|||||||
env = make_sim_env(task_name)
|
env = make_sim_env(task_name)
|
||||||
env_max_reward = env.task.max_reward
|
env_max_reward = env.task.max_reward
|
||||||
|
|
||||||
query_frequency = policy_config['num_queries']
|
query_frequency = num_queries = policy_config['num_queries']
|
||||||
if temporal_agg:
|
if temporal_agg:
|
||||||
query_frequency = 1
|
query_frequency = 1
|
||||||
num_queries = policy_config['num_queries']
|
num_queries = policy_config['num_queries']
|
||||||
@ -335,8 +368,8 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
|
|||||||
plt.ion()
|
plt.ion()
|
||||||
|
|
||||||
### evaluation loop
|
### evaluation loop
|
||||||
if temporal_agg:
|
if temporal_agg or async_query:
|
||||||
all_time_actions = torch.zeros([max_timesteps, max_timesteps+num_queries, 16]).cuda()
|
all_time_actions = torch.zeros([max_timesteps, max_timesteps+num_queries, 16])
|
||||||
|
|
||||||
# qpos_history = torch.zeros((1, max_timesteps, state_dim)).cuda()
|
# qpos_history = torch.zeros((1, max_timesteps, state_dim)).cuda()
|
||||||
qpos_history_raw = np.zeros((max_timesteps, state_dim))
|
qpos_history_raw = np.zeros((max_timesteps, state_dim))
|
||||||
@ -368,17 +401,32 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
|
|||||||
qpos_numpy = np.array(obs['qpos'])
|
qpos_numpy = np.array(obs['qpos'])
|
||||||
qpos_history_raw[t] = qpos_numpy
|
qpos_history_raw[t] = qpos_numpy
|
||||||
qpos = pre_process(qpos_numpy)
|
qpos = pre_process(qpos_numpy)
|
||||||
qpos = torch.from_numpy(qpos).float().cuda().unsqueeze(0)
|
qpos = torch.from_numpy(qpos).float().unsqueeze(0)
|
||||||
# qpos_history[:, t] = qpos
|
# qpos_history[:, t] = qpos
|
||||||
if t % query_frequency == 0:
|
if t % query_frequency == 0:
|
||||||
curr_image = get_image(ts, camera_names, rand_crop_resize=(config['policy_class'] == 'Diffusion'))
|
curr_image = get_image(ts, camera_names, rand_crop_resize=(config['policy_class'] == 'Diffusion'))
|
||||||
# print('get image: ', time.time() - time2)
|
# print('get image: ', time.time() - time2)
|
||||||
|
if not async_query:
|
||||||
|
qpos = qpos.cuda()
|
||||||
|
curr_image = curr_image.cuda()
|
||||||
|
|
||||||
if t == 0:
|
if t == 0:
|
||||||
# warm up
|
print('Warming up')
|
||||||
|
if async_query:
|
||||||
|
num_warmup_steps = 10
|
||||||
|
for _ in tqdm(range(num_warmup_steps)):
|
||||||
|
input_queue.put({'qpos': qpos, 'obs': curr_image, 'query_wall_t': time.time(), 'query_t': t})
|
||||||
|
while action_queue.qsize() == 0:
|
||||||
|
time.sleep(0.01)
|
||||||
|
if action_queue.qsize() > 0:
|
||||||
|
while action_queue.qsize() > 0:
|
||||||
|
res_dict = action_queue.get()
|
||||||
|
query_wall_t = res_dict['query_wall_t']
|
||||||
|
query_t = res_dict['query_t']
|
||||||
|
all_actions = res_dict['action']
|
||||||
|
else:
|
||||||
for _ in range(10):
|
for _ in range(10):
|
||||||
policy(qpos, curr_image)
|
policy(qpos, curr_image)
|
||||||
print('network warm up done')
|
|
||||||
time1 = time.time()
|
time1 = time.time()
|
||||||
|
|
||||||
### query policy
|
### query policy
|
||||||
@ -407,7 +455,7 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
|
|||||||
k = 0.01
|
k = 0.01
|
||||||
exp_weights = np.exp(-k * np.arange(len(actions_for_curr_step)))
|
exp_weights = np.exp(-k * np.arange(len(actions_for_curr_step)))
|
||||||
exp_weights = exp_weights / exp_weights.sum()
|
exp_weights = exp_weights / exp_weights.sum()
|
||||||
exp_weights = torch.from_numpy(exp_weights).cuda().unsqueeze(dim=1)
|
exp_weights = torch.from_numpy(exp_weights).unsqueeze(dim=1)
|
||||||
raw_action = (actions_for_curr_step * exp_weights).sum(dim=0, keepdim=True)
|
raw_action = (actions_for_curr_step * exp_weights).sum(dim=0, keepdim=True)
|
||||||
else:
|
else:
|
||||||
raw_action = all_actions[:, t % query_frequency]
|
raw_action = all_actions[:, t % query_frequency]
|
||||||
@ -415,8 +463,41 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
|
|||||||
# # zero out base actions to avoid overshooting
|
# # zero out base actions to avoid overshooting
|
||||||
# raw_action[0, -2:] = 0
|
# raw_action[0, -2:] = 0
|
||||||
elif config['policy_class'] == "Diffusion":
|
elif config['policy_class'] == "Diffusion":
|
||||||
|
if async_query:
|
||||||
|
freq_mult = 0.8
|
||||||
|
if t == 0 or (t % int(query_frequency*freq_mult) == 0):
|
||||||
|
print(f'{t}: Put')
|
||||||
|
input_queue.put({'qpos': qpos, 'obs': curr_image, 'query_wall_t': time.time(), 'query_t': t})
|
||||||
|
|
||||||
|
# Wait if no action is produced for current step and nothing in the queue
|
||||||
|
no_action = not all_time_actions[:, t].any()
|
||||||
|
no_queue = action_queue.qsize() == 0
|
||||||
|
while no_action and no_queue:
|
||||||
|
time.sleep(0.01)
|
||||||
|
print(f'{t}: wait')
|
||||||
|
no_action = not all_time_actions[:, t].any()
|
||||||
|
no_queue = action_queue.qsize() == 0
|
||||||
|
|
||||||
|
# Get action
|
||||||
|
if action_queue.qsize() > 0:
|
||||||
|
print(f'{t}: Get')
|
||||||
|
while action_queue.qsize() > 0:
|
||||||
|
res_dict = action_queue.get()
|
||||||
|
query_wall_t = res_dict['query_wall_t']
|
||||||
|
query_t = res_dict['query_t']
|
||||||
|
all_actions = res_dict['action']
|
||||||
|
if real_robot:
|
||||||
|
all_actions = torch.cat([all_actions[:, :-BASE_DELAY, :-2], all_actions[:, BASE_DELAY:, -2:]], dim=2)
|
||||||
|
all_time_actions[query_t, query_t:query_t+len(all_actions)] = all_actions
|
||||||
|
|
||||||
|
all_time_curr_actions = all_time_actions[:, t]
|
||||||
|
has_action = all_time_curr_actions.any(dim=1)
|
||||||
|
first_index = torch.arange(len(has_action))[has_action].max().item()
|
||||||
|
# print(first_index)
|
||||||
|
raw_action = all_time_curr_actions[first_index]
|
||||||
|
else:
|
||||||
if t % query_frequency == 0:
|
if t % query_frequency == 0:
|
||||||
all_actions = policy(qpos, curr_image)
|
all_actions = policy(qpos.cuda(), curr_image.cuda())
|
||||||
# if use_actuator_net:
|
# if use_actuator_net:
|
||||||
# collect_base_action(all_actions, norm_episode_all_base_actions)
|
# collect_base_action(all_actions, norm_episode_all_base_actions)
|
||||||
if real_robot:
|
if real_robot:
|
||||||
@ -469,7 +550,7 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
|
|||||||
# print(sleep_time)
|
# print(sleep_time)
|
||||||
time.sleep(sleep_time)
|
time.sleep(sleep_time)
|
||||||
# time.sleep(max(0, DT - duration - culmulated_delay))
|
# time.sleep(max(0, DT - duration - culmulated_delay))
|
||||||
if duration >= DT:
|
if duration >= DT and real_robot:
|
||||||
culmulated_delay += (duration - DT)
|
culmulated_delay += (duration - DT)
|
||||||
print(f'Warning: step duration: {duration:.3f} s at step {t} longer than DT: {DT} s, culmulated delay: {culmulated_delay:.3f} s')
|
print(f'Warning: step duration: {duration:.3f} s at step {t} longer than DT: {DT} s, culmulated delay: {culmulated_delay:.3f} s')
|
||||||
# else:
|
# else:
|
||||||
|
|||||||
@ -65,7 +65,7 @@ class DiffusionPolicy(nn.Module):
|
|||||||
})
|
})
|
||||||
|
|
||||||
nets = nets.float().cuda()
|
nets = nets.float().cuda()
|
||||||
ENABLE_EMA = True
|
ENABLE_EMA = False
|
||||||
if ENABLE_EMA:
|
if ENABLE_EMA:
|
||||||
ema = EMAModel(model=nets, power=self.ema_power)
|
ema = EMAModel(model=nets, power=self.ema_power)
|
||||||
else:
|
else:
|
||||||
@ -125,8 +125,7 @@ class DiffusionPolicy(nn.Module):
|
|||||||
noise_pred = nets['policy']['noise_pred_net'](noisy_actions, timesteps, global_cond=obs_cond)
|
noise_pred = nets['policy']['noise_pred_net'](noisy_actions, timesteps, global_cond=obs_cond)
|
||||||
|
|
||||||
# L2 loss
|
# L2 loss
|
||||||
all_l2 = F.mse_loss(noise_pred, noise, reduction='none')
|
loss = F.mse_loss(noise_pred, noise)
|
||||||
loss = (all_l2 * ~is_pad.unsqueeze(-1)).mean()
|
|
||||||
|
|
||||||
loss_dict = {}
|
loss_dict = {}
|
||||||
loss_dict['l2_loss'] = loss
|
loss_dict['l2_loss'] = loss
|
||||||
|
|||||||
1
utils.py
1
utils.py
@ -93,7 +93,6 @@ class EpisodicDataset(torch.utils.data.Dataset):
|
|||||||
is_pad[action_len:] = 1
|
is_pad[action_len:] = 1
|
||||||
|
|
||||||
padded_action = padded_action[:self.chunk_size]
|
padded_action = padded_action[:self.chunk_size]
|
||||||
is_pad = is_pad[:self.chunk_size]
|
|
||||||
|
|
||||||
# new axis for different cameras
|
# new axis for different cameras
|
||||||
all_cam_images = []
|
all_cam_images = []
|
||||||
|
|||||||
@ -38,9 +38,6 @@ def main(args):
|
|||||||
repr_type = ckpt_name.split('-')[0]
|
repr_type = ckpt_name.split('-')[0]
|
||||||
seed = int(ckpt_name.split('-')[-1][:-3])
|
seed = int(ckpt_name.split('-')[-1][:-3])
|
||||||
|
|
||||||
if 'cotrain' in ckpt_name:
|
|
||||||
repr_type += '_cotrain'
|
|
||||||
|
|
||||||
episode_idxs = [int(name.split('_')[1].split('.')[0]) for name in os.listdir(dataset_dir) if ('.hdf5' in name) and ('features' not in name)]
|
episode_idxs = [int(name.split('_')[1].split('.')[0]) for name in os.listdir(dataset_dir) if ('.hdf5' in name) and ('features' not in name)]
|
||||||
episode_idxs.sort()
|
episode_idxs.sort()
|
||||||
assert len(episode_idxs) == episode_idxs[-1] + 1 # no holes
|
assert len(episode_idxs) == episode_idxs[-1] + 1 # no holes
|
||||||
@ -51,7 +48,6 @@ def main(args):
|
|||||||
for episode_idx in range(num_episodes):
|
for episode_idx in range(num_episodes):
|
||||||
|
|
||||||
# load all images
|
# load all images
|
||||||
print(f'loading data')
|
|
||||||
dataset_path = os.path.join(dataset_dir, f'episode_{episode_idx}.hdf5')
|
dataset_path = os.path.join(dataset_dir, f'episode_{episode_idx}.hdf5')
|
||||||
with h5py.File(dataset_path, 'r') as root:
|
with h5py.File(dataset_path, 'r') as root:
|
||||||
image_dict = {}
|
image_dict = {}
|
||||||
@ -67,7 +63,6 @@ def main(args):
|
|||||||
|
|
||||||
image_dict[cam_name] = image
|
image_dict[cam_name] = image
|
||||||
|
|
||||||
print(f'loading model')
|
|
||||||
# load pretrain nets after cam names are known
|
# load pretrain nets after cam names are known
|
||||||
if not feature_extractors:
|
if not feature_extractors:
|
||||||
for cam_name in camera_names:
|
for cam_name in camera_names:
|
||||||
|
|||||||
@ -97,11 +97,6 @@ def main(args):
|
|||||||
model_name = pathlib.PurePath(model_dir).name
|
model_name = pathlib.PurePath(model_dir).name
|
||||||
seed = int(model_name.split('-')[-1][:-3])
|
seed = int(model_name.split('-')[-1][:-3])
|
||||||
|
|
||||||
repr_type = 'byol'
|
|
||||||
if 'cotrain' in model_name:
|
|
||||||
repr_type += '_cotrain'
|
|
||||||
e() # make sure!
|
|
||||||
|
|
||||||
k = ks[seed]
|
k = ks[seed]
|
||||||
|
|
||||||
if real_robot:
|
if real_robot:
|
||||||
@ -123,7 +118,7 @@ def main(args):
|
|||||||
# Visual feature
|
# Visual feature
|
||||||
all_cam_feature = []
|
all_cam_feature = []
|
||||||
for cam_name in camera_names:
|
for cam_name in camera_names:
|
||||||
feature_dataset_path = os.path.join(dataset_dir, f'{repr_type}_features_seed{seed}_episode_{episode_id}.hdf5')
|
feature_dataset_path = os.path.join(dataset_dir, f'byol_features_seed{seed}_episode_{episode_id}.hdf5')
|
||||||
with h5py.File(feature_dataset_path, 'r') as root:
|
with h5py.File(feature_dataset_path, 'r') as root:
|
||||||
cam_feature = root[f'/features/{cam_name}'][:]
|
cam_feature = root[f'/features/{cam_name}'][:]
|
||||||
all_cam_feature.append(cam_feature)
|
all_cam_feature.append(cam_feature)
|
||||||
|
|||||||
@ -48,11 +48,6 @@ def main(args):
|
|||||||
batch_size = 100
|
batch_size = 100
|
||||||
# TODO ######################
|
# TODO ######################
|
||||||
|
|
||||||
repr_type = 'byol'
|
|
||||||
if 'cotrain' in ckpt_dir:
|
|
||||||
repr_type += '_cotrain'
|
|
||||||
e() # make sure!
|
|
||||||
|
|
||||||
if not os.path.isdir(ckpt_dir):
|
if not os.path.isdir(ckpt_dir):
|
||||||
os.makedirs(ckpt_dir)
|
os.makedirs(ckpt_dir)
|
||||||
|
|
||||||
@ -72,7 +67,7 @@ def main(args):
|
|||||||
camera_names = list(root[f'/observations/images/'].keys())
|
camera_names = list(root[f'/observations/images/'].keys())
|
||||||
|
|
||||||
all_cam_feature = []
|
all_cam_feature = []
|
||||||
feature_dataset_path = os.path.join(dataset_dir, f'{repr_type}_features_seed{seed}_episode_{episode_id}.hdf5')
|
feature_dataset_path = os.path.join(dataset_dir, f'byol_features_seed{seed}_episode_{episode_id}.hdf5')
|
||||||
with h5py.File(feature_dataset_path, 'r') as root:
|
with h5py.File(feature_dataset_path, 'r') as root:
|
||||||
for cam_name in camera_names:
|
for cam_name in camera_names:
|
||||||
cam_feature = root[f'/features/{cam_name}'][:]
|
cam_feature = root[f'/features/{cam_name}'][:]
|
||||||
@ -97,7 +92,7 @@ def main(args):
|
|||||||
action = root['/action'][:]
|
action = root['/action'][:]
|
||||||
|
|
||||||
all_cam_feature = []
|
all_cam_feature = []
|
||||||
feature_dataset_path = os.path.join(dataset_dir, f'{repr_type}_features_seed{seed}_episode_{episode_id}.hdf5')
|
feature_dataset_path = os.path.join(dataset_dir, f'byol_features_seed{seed}_episode_{episode_id}.hdf5')
|
||||||
with h5py.File(feature_dataset_path, 'r') as root:
|
with h5py.File(feature_dataset_path, 'r') as root:
|
||||||
for cam_name in camera_names:
|
for cam_name in camera_names:
|
||||||
cam_feature = root[f'/features/{cam_name}'][:]
|
cam_feature = root[f'/features/{cam_name}'][:]
|
||||||
|
|||||||
在新工单中引用
屏蔽一个用户