trajdl.algorithms.gmvsae module

class trajdl.algorithms.gmvsae.Decoder(emb: Embedding, hidden_size: int, padding_value: float, num_layers: int = 1, dropout: float = 0.0)

Bases: Module

forward(seq: LongTensor, lengths: List[int], init_hidden: Tensor)

init_hidden shape is (B, H)

class trajdl.algorithms.gmvsae.Encoder(embedding_layer: Embedding, hidden_size: int, num_layers: int = 1, dropout: float = 0.0)

Bases: Module

forward(seq: LongTensor, lengths: List[int])

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class trajdl.algorithms.gmvsae.GMVSAE(tokenizer: str | Any, embedding_dim: int, hidden_size: int, mem_num: int, mode: str, num_layers: int = 1, num_neg_samples: int = 64, init_mu_c_pretrained_path: str | None = None, pretrain_ckpt_folder: str | None = None)

Bases: PretrainTrainFramework

abnormal_detect(decoder_seq: LongTensor, decoder_lengths: List[int], decoder_labels: LongTensor, mask: BoolTensor) → Tensor

使用解码器进行序列的异常检测

Parameters:

• decoder_seq (torch.LongTensor) – 解码器输入的序列

• decoder_lengths (List[int]) – 解码器输入的各序列的长度

• decoder_labels (torch.LongTensor) – 解码器需要的输出

• mask (torch.BoolTensor) – 解码器输入序列长度对应的mask矩阵

Returns:

shape is (B,)，表示每条序列的异常分数

Return type:

torch.Tensor

compute_loss(batch: GMVSAESample)

计算损失

Parameters:

batch (GMVSAESample) – 输入样本

Returns:

• loss (torch.Tensor) – shape is (1,)

• batch_size (int) – batch size

decode(init_state: Tensor, decoder_seq: LongTensor, decoder_lengths: List[int], decoder_labels: LongTensor, mask: BoolTensor) → Tensor

init_state: shape is (B, H)

decoder_seq: shape is (B, T + 1), each of seq added BOS decoder_lengths: List[int], encoder_lengths + 1 decoder_labels: shape is (B, T + 1), each of seq added EOS mask: shape is (B, T + 1)

forward(batch: GMVSAESample) → Tensor

推理的逻辑，在预训练阶段和评估阶段的生成结果不同。 1. 预训练阶段是生成隐变量z 2. 评估阶段是生成序列的异常分数。

Parameters:

• batch (GMVSAESample) – 输入样本

• batch_idx (int) – lightning框架使用的batch_idx

Returns:

• z (torch.Tensor) – 当模式为预训练的时候返回这一项，shape is (num_layers, B, H)，隐变量z

• inference_result (torch.Tensor) – 当模式为评估的时候返回这一项，shape is (B,)，是各条序列的异常分数

generate_z(encoder_seq: LongTensor, encoder_lengths: List[int], return_loss: bool = False) → Tuple[Tensor, Tensor, Tensor] | Tensor

这个是编码阶段进行z的生成

Parameters:

• encoder_seq (torch.LongTensor) – 编码器的输入序列，shape是(B, T)

• encoder_lengths (List[int]) – 编码器输入序列各个序列的长度

• return_loss (bool, optional) – 是否返回损失，默认是False

Returns:

• (z, batch_gaussian_loss, batch_uniform_loss) (Tuple[torch.Tensor, torch.Tensor, torch.Tensor]) – 当return_loss为True的时候，再返回两个loss，三个tensor的shape都是(num_layers, B, H)

• z (torch.Tensor) – 当return_loss为False的时候，只返回z，shape是(num_layers, B, H)

init_decoder_state_for_inference(batch_size: int, c_idx: int) → Tensor

init a decoder state for inference

init_from_pretrained_ckpt() → None

当模型是训练模式的时候，从预训练的checkpoint进行参数的加载

property mem_num: int

training_step(batch: GMVSAESample, batch_idx: int)

Here you compute and return the training loss and some additional metrics for e.g. the progress bar or logger.

Parameters:

• batch – The output of your data iterable, normally a DataLoader.

• batch_idx – The index of this batch.

• dataloader_idx – The index of the dataloader that produced this batch. (only if multiple dataloaders used)

Returns:

• Tensor - The loss tensor

• dict - A dictionary which can include any keys, but must include the key 'loss' in the case of automatic optimization.

• None - In automatic optimization, this will skip to the next batch (but is not supported for multi-GPU, TPU, or DeepSpeed). For manual optimization, this has no special meaning, as returning the loss is not required.

In this step you’d normally do the forward pass and calculate the loss for a batch. You can also do fancier things like multiple forward passes or something model specific.

Example:

def training_step(self, batch, batch_idx):
    x, y, z = batch
    out = self.encoder(x)
    loss = self.loss(out, x)
    return loss

To use multiple optimizers, you can switch to ‘manual optimization’ and control their stepping:

def __init__(self):
    super().__init__()
    self.automatic_optimization = False


# Multiple optimizers (e.g.: GANs)
def training_step(self, batch, batch_idx):
    opt1, opt2 = self.optimizers()

    # do training_step with encoder
    ...
    opt1.step()
    # do training_step with decoder
    ...
    opt2.step()

Note

When accumulate_grad_batches > 1, the loss returned here will be automatically normalized by accumulate_grad_batches internally.

validation_step(batch: GMVSAESample, batch_idx: int)

Operates on a single batch of data from the validation set. In this step you’d might generate examples or calculate anything of interest like accuracy.

Parameters:

• batch – The output of your data iterable, normally a DataLoader.

• batch_idx – The index of this batch.

• dataloader_idx – The index of the dataloader that produced this batch. (only if multiple dataloaders used)

Returns:

• Tensor - The loss tensor

• dict - A dictionary. Can include any keys, but must include the key 'loss'.

• None - Skip to the next batch.

# if you have one val dataloader:
def validation_step(self, batch, batch_idx): ...


# if you have multiple val dataloaders:
def validation_step(self, batch, batch_idx, dataloader_idx=0): ...

Examples:

# CASE 1: A single validation dataset
def validation_step(self, batch, batch_idx):
    x, y = batch

    # implement your own
    out = self(x)
    loss = self.loss(out, y)

    # log 6 example images
    # or generated text... or whatever
    sample_imgs = x[:6]
    grid = torchvision.utils.make_grid(sample_imgs)
    self.logger.experiment.add_image('example_images', grid, 0)

    # calculate acc
    labels_hat = torch.argmax(out, dim=1)
    val_acc = torch.sum(y == labels_hat).item() / (len(y) * 1.0)

    # log the outputs!
    self.log_dict({'val_loss': loss, 'val_acc': val_acc})

If you pass in multiple val dataloaders, validation_step() will have an additional argument. We recommend setting the default value of 0 so that you can quickly switch between single and multiple dataloaders.

# CASE 2: multiple validation dataloaders
def validation_step(self, batch, batch_idx, dataloader_idx=0):
    # dataloader_idx tells you which dataset this is.
    ...

Note

If you don’t need to validate you don’t need to implement this method.

Note

When the validation_step() is called, the model has been put in eval mode and PyTorch gradients have been disabled. At the end of validation, the model goes back to training mode and gradients are enabled.

class trajdl.algorithms.gmvsae.LatentSpace(hidden_size: int, num_layers: int, c: int, reduction: str = 'none')

Bases: Module

forward(encoder_state: Tensor)

encoder_state shape is (num_layers, B, H)

get_mean_c(idx: int) → Tensor