2 次代码提交

作者 SHA1 备注 提交日期
0aba9e4f67 real robot not working 2023-12-15 00:09:10 -08:00
a0905ad3b5 async query works in sim 2023-12-14 23:48:42 -08:00
共有 8 个文件被更改,包括 137 次插入 和 182 次删除

查看文件

@ -1,11 +1,13 @@
# Imitation Learning algorithms and Co-training for Mobile ALOHA # ACT: Action Chunking with Transformers
### *New*: [ACT tuning tips](https://docs.google.com/document/d/1FVIZfoALXg_ZkYKaYVh-qOlaXveq5CtvJHXkY25eYhs/edit?usp=sharing)
TL;DR: if your ACT policy is jerky or pauses in the middle of an episode, just train for longer! Success rate and smoothness can improve way after loss plateaus.
#### Project Website: https://mobile-aloha.github.io/ #### Project Website: https://tonyzhaozh.github.io/aloha/
This repo contains the implementation of ACT, Diffusion Policy and VINN, together with 2 simulated environments: This repo contains the implementation of ACT, together with 2 simulated environments:
Transfer Cube and Bimanual Insertion. You can train and evaluate them in sim or real. Transfer Cube and Bimanual Insertion. You can train and evaluate ACT in sim or real.
For real, you would also need to install [Mobile ALOHA](https://github.com/MarkFzp/mobile-aloha). This repo is forked from the [ACT repo](https://github.com/tonyzhaozh/act). For real, you would also need to install [ALOHA](https://github.com/tonyzhaozh/aloha).
### Updates: ### Updates:
You can find all scripted/human demo for simulated environments [here](https://drive.google.com/drive/folders/1gPR03v05S1xiInoVJn7G7VJ9pDCnxq9O?usp=share_link). You can find all scripted/human demo for simulated environments [here](https://drive.google.com/drive/folders/1gPR03v05S1xiInoVJn7G7VJ9pDCnxq9O?usp=share_link).
@ -33,18 +35,17 @@ You can find all scripted/human demo for simulated environments [here](https://d
pip install pyyaml pip install pyyaml
pip install rospkg pip install rospkg
pip install pexpect pip install pexpect
pip install mujoco==2.3.7 pip install mujoco
pip install dm_control==1.0.14 pip install dm_control
pip install opencv-python pip install opencv-python
pip install matplotlib pip install matplotlib
pip install einops pip install einops
pip install packaging pip install packaging
pip install h5py pip install h5py
pip install ipython pip install ipython
pip install ray==2.7.1
cd act/detr && pip install -e . cd act/detr && pip install -e .
- also need to install https://github.com/ARISE-Initiative/robomimic/tree/r2d2 (note the r2d2 branch) for Diffusion Policy by `pip install -e .`
### Example Usages ### Example Usages
To set up a new terminal, run: To set up a new terminal, run:
@ -52,7 +53,7 @@ To set up a new terminal, run:
conda activate aloha conda activate aloha
cd <path to act repo> cd <path to act repo>
### Simulated experiments (LEGACY table-top ALOHA environments) ### Simulated experiments
We use ``sim_transfer_cube_scripted`` task in the examples below. Another option is ``sim_insertion_scripted``. We use ``sim_transfer_cube_scripted`` task in the examples below. Another option is ``sim_insertion_scripted``.
To generated 50 episodes of scripted data, run: To generated 50 episodes of scripted data, run:
@ -60,12 +61,10 @@ To generated 50 episodes of scripted data, run:
python3 record_sim_episodes.py --task_name sim_transfer_cube_scripted --dataset_dir <data save dir> --num_episodes 50 python3 record_sim_episodes.py --task_name sim_transfer_cube_scripted --dataset_dir <data save dir> --num_episodes 50
To can add the flag ``--onscreen_render`` to see real-time rendering. To can add the flag ``--onscreen_render`` to see real-time rendering.
To visualize the simulated episodes after it is collected, run To visualize the episode after it is collected, run
python3 visualize_episodes.py --dataset_dir <data save dir> --episode_idx 0 python3 visualize_episodes.py --dataset_dir <data save dir> --episode_idx 0
Note: to visualize data from the mobile-aloha hardware, use the visualize_episodes.py from https://github.com/MarkFzp/mobile-aloha
To train ACT: To train ACT:
# Transfer Cube task # Transfer Cube task
@ -81,5 +80,3 @@ You can also add ``--onscreen_render`` to see real-time rendering during evaluat
For real-world data where things can be harder to model, train for at least 5000 epochs or 3-4 times the length after the loss has plateaued. For real-world data where things can be harder to model, train for at least 5000 epochs or 3-4 times the length after the loss has plateaued.
Please refer to [tuning tips](https://docs.google.com/document/d/1FVIZfoALXg_ZkYKaYVh-qOlaXveq5CtvJHXkY25eYhs/edit?usp=sharing) for more info. Please refer to [tuning tips](https://docs.google.com/document/d/1FVIZfoALXg_ZkYKaYVh-qOlaXveq5CtvJHXkY25eYhs/edit?usp=sharing) for more info.
### [ACT tuning tips](https://docs.google.com/document/d/1FVIZfoALXg_ZkYKaYVh-qOlaXveq5CtvJHXkY25eYhs/edit?usp=sharing)
TL;DR: if your ACT policy is jerky or pauses in the middle of an episode, just train for longer! Success rate and smoothness can improve way after loss plateaus.

查看文件

@ -205,56 +205,7 @@ CUDA_VISIBLE_DEVICES=1 python3 imitate_episodes.py \
--batch_size 32 --lr 1e-4 --seed 0 \ --batch_size 32 --lr 1e-4 --seed 0 \
--num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000 --num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000
# try chunk size 64, no cotrain
conda activate mobile
export MUJOCO_GL=egl
cd /home/tonyzhao/Research/act-plus-plus
CUDA_VISIBLE_DEVICES=0 python3 imitate_episodes.py \
--task_name aloha_mobile_wipe_wine \
--ckpt_dir /scr/tonyzhao/train_logs/wipe_wine_diffusion_augmentation_chunk64_seed0 \
--policy_class Diffusion --chunk_size 64 \
--batch_size 32 --lr 1e-4 --seed 0 \
--num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000
# chunk 64 with cotrain
conda activate mobile
export MUJOCO_GL=egl
cd /home/tonyzhao/Research/act-plus-plus
CUDA_VISIBLE_DEVICES=1 python3 imitate_episodes.py \
--task_name aloha_mobile_wipe_wine_cotrain \
--ckpt_dir /scr/tonyzhao/train_logs/wipe_wine_cotrain_diffusion_augmentation_chunk64_seed0 \
--policy_class Diffusion --chunk_size 64 \
--batch_size 32 --lr 1e-4 --seed 0 \
--num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000
# chunk 64 with cotrain + EMA
conda activate mobile
export MUJOCO_GL=egl
cd /home/tonyzhao/Research/act-plus-plus
CUDA_VISIBLE_DEVICES=0 python3 imitate_episodes.py \
--task_name aloha_mobile_wipe_wine_2_cotrain \
--ckpt_dir /scr/tonyzhao/train_logs/wipe_wine_cotrain_diffusion_augmentation_chunk64_ema_seed0 \
--policy_class Diffusion --chunk_size 64 \
--batch_size 32 --lr 1e-4 --seed 0 \
--num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000
# chunk 64 with cotrain + EMA + 3e-4
conda activate mobile
export MUJOCO_GL=egl
cd /home/tonyzhao/Research/act-plus-plus
CUDA_VISIBLE_DEVICES=1 python3 imitate_episodes.py \
--task_name aloha_mobile_wipe_wine_2_cotrain \
--ckpt_dir /scr/tonyzhao/train_logs/wipe_wine_cotrain_diffusion_augmentation_chunk64_ema_3e-4_seed0 \
--policy_class Diffusion --chunk_size 64 \
--batch_size 32 --lr 3e-4 --seed 0 \
--num_steps 1000000 --eval_every 1000000 --validate_every 5000 --save_every 5000
######################## VINN ######################## ######################## VINN ########################
@ -400,6 +351,9 @@ python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME} --dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME}
conda activate mobile conda activate mobile
export CUDA_VISIBLE_DEVICES=1 export CUDA_VISIBLE_DEVICES=1
cd /home/tonyzhao/Research/act-plus-plus cd /home/tonyzhao/Research/act-plus-plus
@ -418,83 +372,6 @@ python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/
# push chair task
conda activate mobile
export CUDA_VISIBLE_DEVICES=0
cd /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning
python3 train.py --task aloha_mobile_chair_truncated --cam_name cam_high --seed 0
python3 train.py --task aloha_mobile_chair_truncated --cam_name cam_left_wrist --seed 0
python3 train.py --task aloha_mobile_chair_truncated --cam_name cam_right_wrist --seed 0
cd /home/tonyzhao/Research/act-plus-plus
TASK_NAME=aloha_mobile_chair_truncated
DATA_NAME=aloha_mobile_chair_truncated
python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME}
conda activate mobile
export CUDA_VISIBLE_DEVICES=1
cd /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning
python3 train.py --task aloha_mobile_chair_truncated_cotrain --cam_name cam_high --seed 0
python3 train.py --task aloha_mobile_chair_truncated_cotrain --cam_name cam_left_wrist --seed 0
python3 train.py --task aloha_mobile_chair_truncated_cotrain --cam_name cam_right_wrist --seed 0
cd /home/tonyzhao/Research/act-plus-plus
TASK_NAME=aloha_mobile_chair_truncated_cotrain
DATA_NAME=aloha_mobile_chair_truncated
python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME}
# cache feature again for wipe wine
conda activate mobile
export CUDA_VISIBLE_DEVICES=0
cd /home/tonyzhao/Research/act-plus-plus
TASK_NAME=aloha_mobile_wipe_wine
DATA_NAME=aloha_mobile_wipe_wine
python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME}
cd /home/tonyzhao/Research/act-plus-plus
TASK_NAME=aloha_mobile_wipe_wine_cotrain
DATA_NAME=aloha_mobile_wipe_wine
python3 vinn_cache_feature.py --ckpt_path /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME}
# run on real robot
TASK_NAME=aloha_mobile_chair_truncated
python3 vinn_select_k.py \
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME} \
--ckpt_dir /scr/tonyzhao/train_logs/VINN-eval-${TASK_NAME}-seed-0
python3 vinn_eval.py \
--dataset_dir /scr/tonyzhao/datasets/sim_transfer_cube_scripted \
--model_dir /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
--ckpt_dir /scr/tonyzhao/train_logs/VINN-eval-${TASK_NAME}-seed-0 \
--task_name $TASK_NAME
TASK_NAME=aloha_mobile_chair_truncated
python3 vinn_select_k.py \
--dataset_dir /scr/tonyzhao/mobile_aloha_datasets/${DATA_NAME} \
--ckpt_dir /scr/tonyzhao/train_logs/VINN-eval-${TASK_NAME}-seed-0
python3 vinn_eval.py \
--dataset_dir /scr/tonyzhao/datasets/sim_transfer_cube_scripted \
--model_dir /home/tonyzhao/Research/act-plus-plus/byol_pytorch/examples/lightning/byol-${TASK_NAME}-DUMMY-seed-0.pt \
--ckpt_dir /scr/tonyzhao/train_logs/VINN-eval-${TASK_NAME}-seed-0 \
--task_name $TASK_NAME
# eval on real robot # eval on real robot
conda activate aloha conda activate aloha
@ -517,6 +394,23 @@ python3 vinn_eval.py \
--task_name $TASK_NAME --task_name $TASK_NAME
# debug async model query
conda activate mobile
export MUJOCO_GL=egl
cd /home/tonyzhao/Research/act-plus-plus
CUDA_VISIBLE_DEVICES=0 python3 imitate_episodes.py \
--task_name sim_transfer_cube_scripted \
--ckpt_dir /scr/tonyzhao/train_logs/cube_scripted_diffusion_sweep_6_noEMA_seed1 \
--policy_class Diffusion --chunk_size 32 \
--batch_size 32 --lr 1e-4 --seed 1 \
--num_steps 200000 --eval_every 6000 --validate_every 6000 --save_every 6000 --eval
--------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------
NOTE: chunk size cannot be any number, try before launching NOTE: chunk size cannot be any number, try before launching

查看文件

@ -10,6 +10,8 @@ from tqdm import tqdm
from einops import rearrange from einops import rearrange
import wandb import wandb
import time import time
import ray
from ray.util.queue import Queue
from torchvision import transforms from torchvision import transforms
from constants import FPS from constants import FPS
@ -27,6 +29,8 @@ from sim_env import BOX_POSE
import IPython import IPython
e = IPython.embed e = IPython.embed
ray.init()
def get_auto_index(dataset_dir): def get_auto_index(dataset_dir):
max_idx = 1000 max_idx = 1000
for i in range(max_idx+1): for i in range(max_idx+1):
@ -209,7 +213,7 @@ def get_image(ts, camera_names, rand_crop_resize=False):
curr_image = rearrange(ts.observation['images'][cam_name], 'h w c -> c h w') curr_image = rearrange(ts.observation['images'][cam_name], 'h w c -> c h w')
curr_images.append(curr_image) curr_images.append(curr_image)
curr_image = np.stack(curr_images, axis=0) curr_image = np.stack(curr_images, axis=0)
curr_image = torch.from_numpy(curr_image / 255.0).float().cuda().unsqueeze(0) curr_image = torch.from_numpy(curr_image / 255.0).float().unsqueeze(0)
if rand_crop_resize: if rand_crop_resize:
print('rand crop resize is used!') print('rand crop resize is used!')
@ -225,6 +229,22 @@ def get_image(ts, camera_names, rand_crop_resize=False):
return curr_image return curr_image
@ray.remote(num_gpus=0.1)
class PolicyLoop:
def __init__(self, policy, input_queue, action_queue):
self.input_queue = input_queue
self.action_queue = action_queue
self.policy = policy # Replace with your AI model initialization
def run(self):
while True:
input = self.input_queue.get()
obs = input['obs'].cuda()
qpos = input['qpos'].cuda()
action = self.policy(qpos, obs)
self.action_queue.put({'action': action.cpu(), 'query_wall_t': input['query_wall_t'], 'query_t': input['query_t']})
def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50): def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
set_seed(1000) set_seed(1000)
ckpt_dir = config['ckpt_dir'] ckpt_dir = config['ckpt_dir']
@ -242,6 +262,8 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
actuator_config = config['actuator_config'] actuator_config = config['actuator_config']
use_actuator_net = actuator_config['actuator_network_dir'] is not None use_actuator_net = actuator_config['actuator_network_dir'] is not None
async_query = True
# load policy and stats # load policy and stats
ckpt_path = os.path.join(ckpt_dir, ckpt_name) ckpt_path = os.path.join(ckpt_dir, ckpt_name)
policy = make_policy(policy_class, policy_config) policy = make_policy(policy_class, policy_config)
@ -293,6 +315,17 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
else: else:
post_process = lambda a: a * stats['action_std'] + stats['action_mean'] post_process = lambda a: a * stats['action_std'] + stats['action_mean']
# async
if async_query:
ray.put(policy)
# Create the communication queues
input_queue = Queue()
action_queue = Queue()
# Start the robot AI loop actor
for _ in range(2):
policy_loop = PolicyLoop.remote(policy, input_queue, action_queue)
policy_loop.run.remote()
# load environment # load environment
if real_robot: if real_robot:
from aloha_scripts.robot_utils import move_grippers # requires aloha from aloha_scripts.robot_utils import move_grippers # requires aloha
@ -304,7 +337,7 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
env = make_sim_env(task_name) env = make_sim_env(task_name)
env_max_reward = env.task.max_reward env_max_reward = env.task.max_reward
query_frequency = policy_config['num_queries'] query_frequency = num_queries = policy_config['num_queries']
if temporal_agg: if temporal_agg:
query_frequency = 1 query_frequency = 1
num_queries = policy_config['num_queries'] num_queries = policy_config['num_queries']
@ -335,8 +368,8 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
plt.ion() plt.ion()
### evaluation loop ### evaluation loop
if temporal_agg: if temporal_agg or async_query:
all_time_actions = torch.zeros([max_timesteps, max_timesteps+num_queries, 16]).cuda() all_time_actions = torch.zeros([max_timesteps, max_timesteps+num_queries, 16])
# qpos_history = torch.zeros((1, max_timesteps, state_dim)).cuda() # qpos_history = torch.zeros((1, max_timesteps, state_dim)).cuda()
qpos_history_raw = np.zeros((max_timesteps, state_dim)) qpos_history_raw = np.zeros((max_timesteps, state_dim))
@ -368,18 +401,33 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
qpos_numpy = np.array(obs['qpos']) qpos_numpy = np.array(obs['qpos'])
qpos_history_raw[t] = qpos_numpy qpos_history_raw[t] = qpos_numpy
qpos = pre_process(qpos_numpy) qpos = pre_process(qpos_numpy)
qpos = torch.from_numpy(qpos).float().cuda().unsqueeze(0) qpos = torch.from_numpy(qpos).float().unsqueeze(0)
# qpos_history[:, t] = qpos # qpos_history[:, t] = qpos
if t % query_frequency == 0: if t % query_frequency == 0:
curr_image = get_image(ts, camera_names, rand_crop_resize=(config['policy_class'] == 'Diffusion')) curr_image = get_image(ts, camera_names, rand_crop_resize=(config['policy_class'] == 'Diffusion'))
# print('get image: ', time.time() - time2) # print('get image: ', time.time() - time2)
if not async_query:
qpos = qpos.cuda()
curr_image = curr_image.cuda()
if t == 0: if t == 0:
# warm up print('Warming up')
for _ in range(10): if async_query:
policy(qpos, curr_image) num_warmup_steps = 10
print('network warm up done') for _ in tqdm(range(num_warmup_steps)):
time1 = time.time() input_queue.put({'qpos': qpos, 'obs': curr_image, 'query_wall_t': time.time(), 'query_t': t})
while action_queue.qsize() == 0:
time.sleep(0.01)
if action_queue.qsize() > 0:
while action_queue.qsize() > 0:
res_dict = action_queue.get()
query_wall_t = res_dict['query_wall_t']
query_t = res_dict['query_t']
all_actions = res_dict['action']
else:
for _ in range(10):
policy(qpos, curr_image)
time1 = time.time()
### query policy ### query policy
time3 = time.time() time3 = time.time()
@ -407,7 +455,7 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
k = 0.01 k = 0.01
exp_weights = np.exp(-k * np.arange(len(actions_for_curr_step))) exp_weights = np.exp(-k * np.arange(len(actions_for_curr_step)))
exp_weights = exp_weights / exp_weights.sum() exp_weights = exp_weights / exp_weights.sum()
exp_weights = torch.from_numpy(exp_weights).cuda().unsqueeze(dim=1) exp_weights = torch.from_numpy(exp_weights).unsqueeze(dim=1)
raw_action = (actions_for_curr_step * exp_weights).sum(dim=0, keepdim=True) raw_action = (actions_for_curr_step * exp_weights).sum(dim=0, keepdim=True)
else: else:
raw_action = all_actions[:, t % query_frequency] raw_action = all_actions[:, t % query_frequency]
@ -415,13 +463,46 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
# # zero out base actions to avoid overshooting # # zero out base actions to avoid overshooting
# raw_action[0, -2:] = 0 # raw_action[0, -2:] = 0
elif config['policy_class'] == "Diffusion": elif config['policy_class'] == "Diffusion":
if t % query_frequency == 0: if async_query:
all_actions = policy(qpos, curr_image) freq_mult = 0.8
# if use_actuator_net: if t == 0 or (t % int(query_frequency*freq_mult) == 0):
# collect_base_action(all_actions, norm_episode_all_base_actions) print(f'{t}: Put')
if real_robot: input_queue.put({'qpos': qpos, 'obs': curr_image, 'query_wall_t': time.time(), 'query_t': t})
all_actions = torch.cat([all_actions[:, :-BASE_DELAY, :-2], all_actions[:, BASE_DELAY:, -2:]], dim=2)
raw_action = all_actions[:, t % query_frequency] # Wait if no action is produced for current step and nothing in the queue
no_action = not all_time_actions[:, t].any()
no_queue = action_queue.qsize() == 0
while no_action and no_queue:
time.sleep(0.01)
print(f'{t}: wait')
no_action = not all_time_actions[:, t].any()
no_queue = action_queue.qsize() == 0
# Get action
if action_queue.qsize() > 0:
print(f'{t}: Get')
while action_queue.qsize() > 0:
res_dict = action_queue.get()
query_wall_t = res_dict['query_wall_t']
query_t = res_dict['query_t']
all_actions = res_dict['action']
if real_robot:
all_actions = torch.cat([all_actions[:, :-BASE_DELAY, :-2], all_actions[:, BASE_DELAY:, -2:]], dim=2)
all_time_actions[query_t, query_t:query_t+len(all_actions)] = all_actions
all_time_curr_actions = all_time_actions[:, t]
has_action = all_time_curr_actions.any(dim=1)
first_index = torch.arange(len(has_action))[has_action].max().item()
# print(first_index)
raw_action = all_time_curr_actions[first_index]
else:
if t % query_frequency == 0:
all_actions = policy(qpos.cuda(), curr_image.cuda())
# if use_actuator_net:
# collect_base_action(all_actions, norm_episode_all_base_actions)
if real_robot:
all_actions = torch.cat([all_actions[:, :-BASE_DELAY, :-2], all_actions[:, BASE_DELAY:, -2:]], dim=2)
raw_action = all_actions[:, t % query_frequency]
elif config['policy_class'] == "CNNMLP": elif config['policy_class'] == "CNNMLP":
raw_action = policy(qpos, curr_image) raw_action = policy(qpos, curr_image)
all_actions = raw_action.unsqueeze(0) all_actions = raw_action.unsqueeze(0)
@ -469,7 +550,7 @@ def eval_bc(config, ckpt_name, save_episode=True, num_rollouts=50):
# print(sleep_time) # print(sleep_time)
time.sleep(sleep_time) time.sleep(sleep_time)
# time.sleep(max(0, DT - duration - culmulated_delay)) # time.sleep(max(0, DT - duration - culmulated_delay))
if duration >= DT: if duration >= DT and real_robot:
culmulated_delay += (duration - DT) culmulated_delay += (duration - DT)
print(f'Warning: step duration: {duration:.3f} s at step {t} longer than DT: {DT} s, culmulated delay: {culmulated_delay:.3f} s') print(f'Warning: step duration: {duration:.3f} s at step {t} longer than DT: {DT} s, culmulated delay: {culmulated_delay:.3f} s')
# else: # else:

查看文件

@ -65,7 +65,7 @@ class DiffusionPolicy(nn.Module):
}) })
nets = nets.float().cuda() nets = nets.float().cuda()
ENABLE_EMA = True ENABLE_EMA = False
if ENABLE_EMA: if ENABLE_EMA:
ema = EMAModel(model=nets, power=self.ema_power) ema = EMAModel(model=nets, power=self.ema_power)
else: else:
@ -125,8 +125,7 @@ class DiffusionPolicy(nn.Module):
noise_pred = nets['policy']['noise_pred_net'](noisy_actions, timesteps, global_cond=obs_cond) noise_pred = nets['policy']['noise_pred_net'](noisy_actions, timesteps, global_cond=obs_cond)
# L2 loss # L2 loss
all_l2 = F.mse_loss(noise_pred, noise, reduction='none') loss = F.mse_loss(noise_pred, noise)
loss = (all_l2 * ~is_pad.unsqueeze(-1)).mean()
loss_dict = {} loss_dict = {}
loss_dict['l2_loss'] = loss loss_dict['l2_loss'] = loss

查看文件

@ -93,7 +93,6 @@ class EpisodicDataset(torch.utils.data.Dataset):
is_pad[action_len:] = 1 is_pad[action_len:] = 1
padded_action = padded_action[:self.chunk_size] padded_action = padded_action[:self.chunk_size]
is_pad = is_pad[:self.chunk_size]
# new axis for different cameras # new axis for different cameras
all_cam_images = [] all_cam_images = []

查看文件

@ -38,9 +38,6 @@ def main(args):
repr_type = ckpt_name.split('-')[0] repr_type = ckpt_name.split('-')[0]
seed = int(ckpt_name.split('-')[-1][:-3]) seed = int(ckpt_name.split('-')[-1][:-3])
if 'cotrain' in ckpt_name:
repr_type += '_cotrain'
episode_idxs = [int(name.split('_')[1].split('.')[0]) for name in os.listdir(dataset_dir) if ('.hdf5' in name) and ('features' not in name)] episode_idxs = [int(name.split('_')[1].split('.')[0]) for name in os.listdir(dataset_dir) if ('.hdf5' in name) and ('features' not in name)]
episode_idxs.sort() episode_idxs.sort()
assert len(episode_idxs) == episode_idxs[-1] + 1 # no holes assert len(episode_idxs) == episode_idxs[-1] + 1 # no holes
@ -51,7 +48,6 @@ def main(args):
for episode_idx in range(num_episodes): for episode_idx in range(num_episodes):
# load all images # load all images
print(f'loading data')
dataset_path = os.path.join(dataset_dir, f'episode_{episode_idx}.hdf5') dataset_path = os.path.join(dataset_dir, f'episode_{episode_idx}.hdf5')
with h5py.File(dataset_path, 'r') as root: with h5py.File(dataset_path, 'r') as root:
image_dict = {} image_dict = {}
@ -67,7 +63,6 @@ def main(args):
image_dict[cam_name] = image image_dict[cam_name] = image
print(f'loading model')
# load pretrain nets after cam names are known # load pretrain nets after cam names are known
if not feature_extractors: if not feature_extractors:
for cam_name in camera_names: for cam_name in camera_names:

查看文件

@ -97,11 +97,6 @@ def main(args):
model_name = pathlib.PurePath(model_dir).name model_name = pathlib.PurePath(model_dir).name
seed = int(model_name.split('-')[-1][:-3]) seed = int(model_name.split('-')[-1][:-3])
repr_type = 'byol'
if 'cotrain' in model_name:
repr_type += '_cotrain'
e() # make sure!
k = ks[seed] k = ks[seed]
if real_robot: if real_robot:
@ -123,7 +118,7 @@ def main(args):
# Visual feature # Visual feature
all_cam_feature = [] all_cam_feature = []
for cam_name in camera_names: for cam_name in camera_names:
feature_dataset_path = os.path.join(dataset_dir, f'{repr_type}_features_seed{seed}_episode_{episode_id}.hdf5') feature_dataset_path = os.path.join(dataset_dir, f'byol_features_seed{seed}_episode_{episode_id}.hdf5')
with h5py.File(feature_dataset_path, 'r') as root: with h5py.File(feature_dataset_path, 'r') as root:
cam_feature = root[f'/features/{cam_name}'][:] cam_feature = root[f'/features/{cam_name}'][:]
all_cam_feature.append(cam_feature) all_cam_feature.append(cam_feature)

查看文件

@ -48,11 +48,6 @@ def main(args):
batch_size = 100 batch_size = 100
# TODO ###################### # TODO ######################
repr_type = 'byol'
if 'cotrain' in ckpt_dir:
repr_type += '_cotrain'
e() # make sure!
if not os.path.isdir(ckpt_dir): if not os.path.isdir(ckpt_dir):
os.makedirs(ckpt_dir) os.makedirs(ckpt_dir)
@ -72,7 +67,7 @@ def main(args):
camera_names = list(root[f'/observations/images/'].keys()) camera_names = list(root[f'/observations/images/'].keys())
all_cam_feature = [] all_cam_feature = []
feature_dataset_path = os.path.join(dataset_dir, f'{repr_type}_features_seed{seed}_episode_{episode_id}.hdf5') feature_dataset_path = os.path.join(dataset_dir, f'byol_features_seed{seed}_episode_{episode_id}.hdf5')
with h5py.File(feature_dataset_path, 'r') as root: with h5py.File(feature_dataset_path, 'r') as root:
for cam_name in camera_names: for cam_name in camera_names:
cam_feature = root[f'/features/{cam_name}'][:] cam_feature = root[f'/features/{cam_name}'][:]
@ -97,7 +92,7 @@ def main(args):
action = root['/action'][:] action = root['/action'][:]
all_cam_feature = [] all_cam_feature = []
feature_dataset_path = os.path.join(dataset_dir, f'{repr_type}_features_seed{seed}_episode_{episode_id}.hdf5') feature_dataset_path = os.path.join(dataset_dir, f'byol_features_seed{seed}_episode_{episode_id}.hdf5')
with h5py.File(feature_dataset_path, 'r') as root: with h5py.File(feature_dataset_path, 'r') as root:
for cam_name in camera_names: for cam_name in camera_names:
cam_feature = root[f'/features/{cam_name}'][:] cam_feature = root[f'/features/{cam_name}'][:]