torchebm.utils.replay_buffer ¶
Replay buffer of persistent sampling chains.
ReplayBuffer ¶
Bases: Module
Replay buffer of persistent sampling chains.
Holds the last capacity samples in a FIFO and serves start points for new chains: each draw is a uniformly random stored sample, replaced with probability new_sample_ratio by a fresh \(\mathcal{N}(0, \sigma^2 I)\) sample so the chains keep exploring. This is persistent contrastive divergence (Tieleman, 2008) with the sample replay buffer of Du and Mordatch (2019). The storage starts filled with the same noise, so the first draws are fresh chains.
Distributed runs keep the buffer rank-local: every rank holds independent chains and no default path issues a collective. Chains are exchanged only through mix_across_ranks. Checkpoints are rank-local too: save one file per rank and restore with the same world size. Keep this state out of torch.distributed.checkpoint, whose planner deduplicates non-sharded tensors and would keep rank 0's chains only.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
capacity | int | Number of stored samples. | required |
sample_shape | Tuple[int, ...] | Shape of one sample. | required |
new_sample_ratio | float | Probability that a drawn sample is replaced by a fresh one. | 0.05 |
init_noise_scale | float | \(\sigma\) of the fresh samples and of the initial storage. | 1.0 |
device | Optional[device] | Storage device. | None |
dtype | Optional[dtype] | Storage dtype. | None |
generator | Optional[Generator] | RNG for the initial storage; the global RNG when | None |
Example
The buffer of JEM (Grathwohl et al., 2020) on CIFAR-10: 10000 samples, 5% of the chains reinitialized, 20 SGLD steps per draw of 64:
Source code in torchebm/utils/replay_buffer.py
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 | |
add(samples) ¶
Append a batch, overwriting the oldest entries.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
samples | Tensor | Tensor of shape (n, *sample_shape); detached and moved to the storage device and dtype. A batch larger than the capacity keeps its last | required |
Raises:
| Type | Description |
|---|---|
ValueError | If the trailing shape differs from |
Source code in torchebm/utils/replay_buffer.py
mix_across_ranks(process_group=None, *, generator=None) ¶
Redistribute the pooled chains uniformly across ranks.
A collective: every rank in the group calls it together, outside the training step. The pooled storage is re-dealt with one permutation drawn on rank 0 and broadcast, so no chain is duplicated or lost and per-rank generators cannot desynchronize the shuffle. Costs one all-gather of the full storage (transiently the world size times the buffer memory) plus a small broadcast, and requires equal capacity on every rank. The write position is unchanged. No-op in single-process runs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
process_group | Optional[ProcessGroup] | Group to mix over; the default group when | None |
generator | Optional[Generator] | CPU generator for the permutation; used on rank 0 only. | None |
Source code in torchebm/utils/replay_buffer.py
sample(batch_size, *, generator=None) ¶
Draw batch_size start points.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
batch_size | int | Number of samples. | required |
generator | Optional[Generator] | RNG for the index draw, the replacement mask and the fresh samples; the global RNG when | None |
Returns:
| Type | Description |
|---|---|
Tensor | Tensor of shape (batch_size, *sample_shape), independent of the |
Tensor | storage. |