title: Segmentation Fault when using Pytorch with Pytorch-Lightning ROC metrics

body:
For a machine learning task, I am transforming two tensors, as shown in code snippet (1), in order to make them fit to the expected format from pytorch-lightning's ROC metric. The two resulting tensors y_test (gt_classes) and y_score (detection_scores) are then passed to the function calc_multi_class in code snippet (2) in which I make use of the Receiver Operating Characteristic Metric from pytorch-lightning: fpr, tpr, thr = roc(y_score, y_test). `
Since I integrated pytorch-lightning and make use of their ROC function I receive the error message 'Segmentation fault' as shown in the section bellow.
I am using Python 3.7 and Pytorch 1.7, Pytorch-Lightning 1.1.2, with Cuda 10.2
    Part (1)
    ...
    # sort by class and by confidence value
    # gt_classes = labels[:, 0].int()
    gt_classes_sorted = labels[labels[:, 0].sort(descending=False)[1]]
    gt_classes = gt_classes_sorted[:, 0].int()
    # x = x[x[:, 4].argsort(descending=True)]
    detections_sorted = detection[detection[:, 4].sort(descending=True)[1]]
    det_classes_sorted = detections_sorted[:, 5].int()

    # filter for scores
    input = torch.empty(gt_classes.numel(), self.nc)
    detection_scores = torch.ones_like(input) * 0.01

    # check if no scores
    if detections_sorted.nelement() > 0 and det_classes_sorted.nelement() > 0:
        counter = 0
        for j, dc in enumerate(det_classes_sorted):
            if counter < gt_classes.numel():
                for i, gc in enumerate(gt_classes):
                    # classes gc equals dc classes
                    if gc == dc:
                        # conf value
                        print('Found detection score {}'.format(detections_sorted[j, 4]))
                        detection_scores[i, gc] = detections_sorted[j, 4]
                        counter = counter + 1
                    else:
                        print('Other detection score {}'.format(detections_sorted[j, 4]))
                        detection_scores[i, dc] = detections_sorted[j, 4] #conf value
    #print('Result classes {} scores {}'.format(gt_classes, detection_scores))

    gt_classes, detection_scores = gt_classes.to(self.device), detection_scores.to(self.device)
    return gt_classes, detection_scores

   Part (2)
    def calc_multi_class_(self, y_test, y_score, num_classes=3):
    """ Use of Pytorch Lightning Metrics Library for ROC"""

    fprs = dict()
    tprs = dict()
    roc_aucs = dict()
    thresholds = dict()

    # torch to cuda device
    if isinstance(y_test, np.ndarray):
        y_test = torch.from_numpy(y_test)
        y_test = y_test.cpu()

    if isinstance(y_score, np.ndarray):
        y_score = torch.from_numpy(y_score)
        y_score = y_score.cpu()

    if y_score.nelement() > 0 and torch.max(y_score) > 0.2:
        roc = classification.ROC(num_classes=num_classes)  # no neg classes
        fpr, tpr, thr = roc(y_score, y_test)

        # convert multi-dim tensor to numpy ndarray
        for i in range(num_classes):
            thresholds[i] = thr[i].numpy()
            fprs[i], tprs[i] = fpr[i].numpy(), tpr[i].numpy()
            roc_aucs[i] = auc(fprs[i], tprs[i])

    return fprs, tprs, roc_aucs, thresholds

As I am still quite new to Pytorch, I tried to figure out where it came from and had a look if I set the tensors wrong, but couldn't figure it out yet. If anyone has a hint or an idea, this would be very helpful!!
The resulting error message is the following:
    Result classes tensor([1], device='cuda:0', dtype=torch.int32) scores tensor([[0.01000, 0.01000, 0.01000]], device='cuda:0')
           Class      Images     Targets           P           R      mAP@.5  mAP@.5:.95: 100%|██████████| 58/58 [00:13<00:00,  4.41it/s]
             all         928         489           0           0    3.57e-06    5.65e-07

/home/jm0365/.direnv/python-3.7.3/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py:49: UserWarning: Metric ROC will save all targets and predictions in buffer. For large datasets this may lead to large memory footprint. warnings.warn(*args, **kwargs) Segmentation fault


title: How does PyTorch DataLoader interact with a PyTorch dataset to transform batches?

body:
I'm creating a custom dataset for NLP-related tasks.
In the PyTorch custom datast tutorial, we see that the __getitem__() method leaves room for a transform before it returns a sample:
def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        img_name = os.path.join(self.root_dir,
                                self.landmarks_frame.iloc[idx, 0])
        image = io.imread(img_name)

        ### SOME DATA MANIPULATION HERE ###

        sample = {'image': image, 'landmarks': landmarks}
        if self.transform:
            sample = self.transform(sample)

        return sample

However, the code here:
        if torch.is_tensor(idx):
            idx = idx.tolist()

implies that multiple items should be able to be retrieved at a time which leaves me wondering:

How does that transform work on multiple items? Take the custom transforms in the tutorial for example. They do not look like they could be applied to a batch of samples in a single call.

Related, how does a DataLoader retrieve a batch of multiple samples in parallel and apply said transform if the transform can only be applied to a single sample?


title: How can I resume training pl.Trainer after interruption?

body:
I have Model and Trainer pytorch-lightning objects which are initialized as follows:
checkpoint_callback = ModelCheckpoint(
    filepath=os.path.join('experiments', experiment_name, '{epoch}-{avg_valid_iou:.4f}'),
    save_top_k=1,
    verbose=True,
    monitor='avg_valid_iou',
    mode='max',
    prefix=''
)
model = nn.DataParallel (FaultNetPL(batch_size=5)).cuda()
trainer = Trainer( checkpoint_callback=checkpoint_callback,
                  max_epochs=500,gpus=1,
                  logger=logger)

Then I start training using:
trainer.fit(model)

But  training was interrupted and now I would like to resume it using checkpoint from N-th iteration
So i tried initialize model and trainer as:
model = FaultNetPL.load_from_checkpoint('experiments/VNET/epoch=77-avg_valid_iou=0.7604.ckpt',batch_size=5)
trainer = Trainer(resume_from_checkpoint = 'epoch=77-avg_valid_iou=0.7604.ckpt',
                  checkpoint_callback=checkpoint_callback,
                  max_epochs=500,gpus=1,
                  logger=logger)

But again and again training starting from scratch (from 0th epoch and huge error). What did I miss?


title: DataLoader - shuffle implicit pairs

body:
Is there a way to handle the DataLoader as a list ? The idea is that I want to shuffle implicit pairs of images, without setting the shuffling into True

Basically, I have for example 10 scenes, each containing let's say 100 sequences, so they are represented inside the directory as

'1_1.png', '1_2.png', '1_3.png', '....., '2_1.png', '2_2.png', '2_3.png', ...., '3_1.png', '3_2.png', '3_3.png', ..., ...., '10_1.png', '10_2.png', '10_3.png', ...

I don't want complete shuffling of data, what I want simply is to shuffle but keeping pairs, so they are represented in the data loader as

[ '1_3.png', '1_4.png', '2_2.png', '2_3.png', '10_1.png', '10_2.png', '1_2.png', '1_3.png', ...]
and so on

Please have a look at this question which I have already asked on Stack Overflow concerning shuffling array of implicit pairs, where you can understand what I mean

As an example:
if this is a list

L = [['1_1'],['1_2'],['1_3'],['1_4'],['1_5'],['1_6'],['2_1'],['2_2'],['2_3'],['2_4'],['2_5'],['2_6'],['3_1'],['3_2'],['3_3'],['3_4'],['3_5'],['3_6']]

then this is the output

[['1_2'], ['1_3'], ['2_1'], ['2_2'], ['2_4'], ['2_5'],
 ['2_2'], ['2_3'], ['1_3'], ['1_4'], ['3_4'], ['3_5'],
 ['3_3'], ['3_4'], ['3_2'], ['3_3'], ['1_6'], ['2_1'],
 ['2_5'], ['2_6'], ['2_6'], ['3_1'], ['1_4'], ['1_5'],
 ['1_1'], ['1_2'], ['2_3'], ['2_4'], ['1_5'], ['1_6'],
 ['3_1'], ['3_2'], ['3_5'], ['3_6']]

I want to achieve the same for a DataLoader

The main idea, is that I want to train my network on sequential frames, but it doesn't have to be the complete sequence, but at least I need each step, two sequences are there


title: Pytorch: how to repeat a parameter matrix into a bigger one along both dimensions?

body:
What is the simplest syntax to transform 2D parameter tensor

A   B
C   D

into

A  A  B  B
A  A  B  B
C  C  D  D
C  C  D  D

Note they are parameter tensors, so I need autograd to back propagate gradient from latter into former.
Thanks!


title: why this model fail to converge without manual weight initialization step (pytorch linear regression)

body:
I am trying to create a most simple 1 linear layer network to fit a linear regression. Just to help myself better understand how Pytorch works. However, I encountered a strange issue with the model training.

in my model’s init() method, I have to add a manual initialization step(shown below) to have the model quickly converge to my regression function. (the weight value 2, 3 are random number, I could put any value here and the model will still converge)

self.layer1.weight = torch.nn.Parameter(torch.Tensor([2, 3]))

Without this line, the model never converge, the training loss just randomly oscillates in the range of hundreds of thousands. With this line, it quickly decreases to near 1.

I have postulated that it is because default initial weight parameters were too small if I do not initialize them to be far away from zero. Then I changed the initial values and found out the convergence always work as long as I have this line, the exact value I set does not matter. I tried using debugger to step through this line and use vars() method to examine the attributes of self.layer1, the only thing changed is the weights, everything else remains the same. Could someone explain what is going on behind the scene here? Thanks.

My entire script:

import torch
import numpy as np

class Net(torch.nn.Module):

    def __init__(self, input_dim, output_dim):
        super(Net, self).__init__()
        self.layer1 = torch.nn.Linear(input_dim, output_dim, bias=False)

        # the line below is the strange initialization required to
        # have the model converge.
        self.layer1.weight = torch.nn.Parameter(torch.Tensor([2, 3]))

    def forward(self, x):
        x = self.layer1(x)
        return x

# generate data using the linear regression setup y = 5 * x1 + 3 * x2

sample_size = 10000
input_dim = 2
output_dim = 1
epoch = 30
bs = 100

data = np.random.randn(sample_size, 3)
data[:, :2] = data[:, :2] * 100
# add a normal noise term
data[:, 2] = 5 * data[:, 0] + 3 * data[:, 1] + np.random.randn(sample_size)
data = torch.Tensor(data)
train_x = data[:, :input_dim]
train_y = data[:, input_dim]

net = Net(input_dim, output_dim)
criterion = torch.nn.MSELoss()
optimizer = torch.optim.RMSprop(net.parameters(), lr=.01)

for i in range(epoch):

    batch = 0
    while batch * bs < train_x.shape[0]:

        batch_x = train_x[batch * bs : (batch + 1) * bs, :]
        batch_y = train_y[batch * bs : (batch + 1) * bs]

        pred_y = net.forward(batch_x)
        loss = criterion(pred_y, batch_y)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        if batch % 100 == 0:
            #print(f"{i} {batch} {loss}")
            print(net.layer1.weight)
        batch += 1


title: Using Bert to convert Vectors to Word in pytorch

body:
I am using Bert for sentence embedding and I would like to then take that embedding and convert back to word. How would I go about doing this? Is this even possible?

Any help is very much appreciated!

I have tried googling around but couldn't find much because of how new bert is.


title: PyTorch: How to write a neural network that only returns the weights?

body:
I'm training a neural network that learns some weights and based on those weights, I compute transformations that produce the predicted model in combination with the weights. My network doesn't learn properly and therefore I'm writing a different network that does nothing but returning the weights independent from the input x (after normalization with softmax and transpose). This way, I want to find out whether the problem lies in the network or in the transformation estimation outside the network. But this doesn't work. This is what I've got.

class DoNothingNet(torch.nn.Module):
    def __init__(self, n_vertices=6890, n_joints=14):
        super(DoNothingNet, self).__init__()
        self.weights = nn.parameter.Parameter(torch.randn(n_vertices, n_joints))

    def forward(self, x, indices):
        self.weights = F.softmax(self.weights, dim=1)
        return self.weights.transpose(0,1)

But the line self.weights = F.softmax(self.weights, dim=1) doesn't work and produces the error TypeError: cannot assign 'torch.cuda.FloatTensor' as parameter 'weights' (torch.nn.Parameter or None expected). How do I fix this? And does the code even make sense?


title: Getting 'tensor is not a torch image' for data type

body:
As the question suggests, I'm trying to convert images to tensor.

X, y = train_sequence[idx]

images = Variable(torch.from_numpy(X)).to(device) # [batch, channel, H, W]
masks = Variable(torch.from_numpy(y)).to(device)
print(type(images)) ## Output: <class 'torch.Tensor'>

images = transforms.Normalize((0.5, 0.5, 0.5, 0.5, 0.5), (0.5, 0.5, 0.5,0.5, 0.5))(images)
masks =  transforms.Normalize((0.5), (0.5))(masks)

But I get the error at
 ---> 19         images = transforms.Normalize((0.5, 0.5, 0.5, 0.5, 0.5), (0.5, 0.5, 0.5,0.5, 0.5))(images)

TypeError: tensor is not a torch image.


title: BERT token importance measuring issue. Grad is none

body:
I am trying to measure token importance for BERT via comparing token embedding grad value. So, to get the grad, I've copied the 2.8.0 forward of BertModel and changed it a bit:

huggingface transformers 2.8.0 BERT https://github.com/huggingface/transformers/blob/11c3257a18c4b5e1a3c1746eefd96f180358397b/src/transformers/modeling_bert.py

Code:

        embedding_output = self.embeddings(
            input_ids=input_ids, position_ids=position_ids, token_type_ids=token_type_ids, inputs_embeds=inputs_embeds
        )
        embedding_output = embedding_output.requires_grad_(True) # my code
        encoder_outputs = self.encoder(
            embedding_output,
            attention_mask=extended_attention_mask,
            head_mask=head_mask,
            encoder_hidden_states=encoder_hidden_states,
            encoder_attention_mask=encoder_extended_attention_mask,
        )
        sequence_output = encoder_outputs[0]
        sequence_output.mean().backward() # my code
        assert(embedding_output.grad is not None) # my code

Colab link: https://colab.research.google.com/drive/1MggBUaDWAAZNuXbTDM11E8jvdMGEkuRD
But it gives assertion error. I do not understand why and it seems to be a bug for me.
Please, help!


title: torchtext IMDB Dataset Contains Only Positive Labels in Training Set

body:
I am currently trying to use the pre-loaded IMDB dataset for sentiment analysis in pytorch.

I have currently built train and test iterators with the following code:

train_iter, test_iter torchtext.datasets.IMDB.iters(batch_size = 4)

But then when I print out a few batches of labels from the train iterator using the following code, I noticed that only positive labels exist in the training set:

j = 20            #print the first 20 batches of labels

for i, val in enumerate(train_iter):
    print(val.label)
    if i == j:
        break

This code returns the following:

tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 1, 1, 1])

My suspicions are confirmed by checking which unique values exist in the label set for the training data:

for batch in train_iter:
    for val in batch.label:
        if val.item() == 0:
            print(True)

As it turns out, true is never printed when this statement is ran.

Now, if I check the test data that I created, the data appears to be equally balanced between negative and positive labels. Here is the code used to print the first 20 test batches:

j = 20
for i, val in enumerate(test_iter):
    print(val.label)
    if i == j:
        break

And heres the output from the above code:

tensor([1, 1, 1, 1])
tensor([0, 0, 0, 1])
tensor([1, 0, 0, 0])
tensor([0, 0, 0, 0])
tensor([0, 1, 0, 0])
tensor([1, 1, 0, 1])
tensor([0, 0, 0, 1])
tensor([1, 0, 0, 0])
tensor([1, 1, 1, 0])
tensor([1, 0, 0, 1])
tensor([0, 0, 0, 0])
tensor([1, 1, 0, 0])
tensor([0, 0, 0, 0])
tensor([1, 1, 0, 0])
tensor([0, 0, 0, 1])
tensor([1, 0, 0, 0])
tensor([0, 1, 1, 1])
tensor([1, 1, 0, 0])
tensor([1, 1, 1, 1])
tensor([0, 0, 0, 0])
tensor([1, 1, 1, 0])

What's going on here?


title: How did Tensorflow process images of ImageNet in Tfrecord format when training resnet?

body:
emm, converting imagenet to Tfrecord format is quite complicated, so I downloaded the processed Tfrecord imagenet from somewhere.

I applied it to resnet34 in tensorflow with parameters from Pytorch, but found that the accuracy is just 55%, too low. I guess the reason may be the different methods processing imagenet between Pytorch.models and this Tfrecord. And a nice bro has told me the way how Pytorch processing data, but I still need to know how tensorflow processing it.

I found that the values of Tfrecord pictures range from -1 to 1, can you tell me the processing method of this Tfrecord so that I can try to improve the accuracy?

Thanks a lot! I am just too new_flesh, your kind help is so so important


title: What makes BertGeneration and/or RobertaForCausalLM causal models? Where does the causal attention masking happen?

body:
I am trying to use RobertaForCausalLM and/or BertGeneration for causal language modelling / next-word-prediction / left-to-right prediction. I can't seem to figure out where the causal masking is happening? I want to train teacher forcing with the ground-truth labels, but no information from future tokens to be included in the attention mechanism. For that I thought the model would need causal attention masking, but I don't see it being applied anywhere...
If anyone could point me to where this might be happening or why it is unnecessary, that would be helpful.
Thanks!


title: Pytorch training error PermissionError: [WinError 5] Access Denied

body:
Recently, I have installed Pytorch 1.7.1, cuda11.1 and python 3.8.5 on Win10. However, after training for a few epochs, errors like "PermissionError: [WinError 5]" always happened. I have tried to revise the python jurisdiction，but it failed. Has anyone met such issuses?
Below is the error info:

  File "D:\Pycharm\Anaconda\lib\site-packages\tqdm\std.py", line 1165, in __iter__
    for obj in iterable:
  File "D:\Pycharm\Anaconda\lib\site-packages\torch\utils\data\dataloader.py", line 352, in __iter__
    return self._get_iterator()
  File "D:\Pycharm\Anaconda\lib\site-packages\torch\utils\data\dataloader.py", line 294, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "D:\Pycharm\Anaconda\lib\site-packages\torch\utils\data\dataloader.py", line 801, in __init__
    w.start()
  File "D:\Pycharm\Anaconda\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "D:\Pycharm\Anaconda\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "D:\Pycharm\Anaconda\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "D:\Pycharm\Anaconda\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "D:\Pycharm\Anaconda\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "D:\Pycharm\Anaconda\lib\multiprocessing\synchronize.py", line 104, in __getstate__
    h = context.get_spawning_popen().duplicate_for_child(sl.handle)
  File "D:\Pycharm\Anaconda\lib\multiprocessing\popen_spawn_win32.py", line 99, in duplicate_for_child
    return reduction.duplicate(handle, self.sentinel)
  File "D:\Pycharm\Anaconda\lib\multiprocessing\reduction.py", line 79, in duplicate
    return _winapi.DuplicateHandle(
PermissionError: [WinError 5] Access Denied。


title: How to vectorize indexing and computation when indexed tensors are different dimensions?

body:
I'm trying to vectorize the following for-loop in Pytorch. I'd be happy with just vectorizing the inner for-loop, but doing the whole batch would also be awesome.
# B: the batch size
# N: the number of training examples
# dim: the dimension of each feature vector
# K: the number of discrete labels. each vector has a single label
# delta: margin for hinge loss

batch_data = torch.tensor(...)  # Tensor of shape [B x N x d]
batch_labels = torch.tensor(...)  # Tensor of shape [B x N x 1], each element is one of K labels (ints)

batch_losses = []  # Ultimately should be [B x 1]
batch_centroids = []  # Ultimately should be [B x K_i x dim]
for i in range(B):
    centroids = []  # Keep track of the means for each class.
    classes = torch.unique(labels)  # Get the unique labels for the classes.

    # NOTE: The number of classes K for each item in the batch might actually
    # be different. This may complicate batch-level operations.

    total_loss = 0

    # For each class independently. This is the part I want to vectorize.
    for cl in classes:
        # Take the subset of training examples with that label.
        subset = data[torch.where(labels == cl)]

        # Find the centroid of that subset.
        centroid = subset.mean(dim=0)
        centroids.append(centroid)

        # Get the distance between each point in the subset and the centroid.
        dists = subset - centroid
        norm = torch.linalg.norm(dists, dim=1)

        # The loss is the mean of the hinge loss across the subset.
        margin = norm - delta
        hinge = torch.clamp(margin, min=0.0) ** 2

        total_loss += hinge.mean()

    # Keep track of everything. If it's too hard to keep track of centroids, that's also OK.
    loss = total_loss.mean()
    batch_losses.append(loss)
    batch_centroids.append(centroids)

I've been scratching my head on how to deal with the irregularly sized tensors. The number of classes in each batch K_i is different, and the size of each subset is different.


title: Fast.ai Course 2020: HTTPError: 401 Client Error: PermissionDenied for url

body:
I am going through the fast.ai course (version 2020) and am currently in lesson 2.
When trying to use the azure key, I always get this error and I can't figure out what to do:
#To download images with Bing Image Search, sign up at Microsoft Azure for a free account. You will be given a key, which you can copy and enter in a cell as follows (replacing 'XXX' with your key and executing it):
key = os.environ.get('AZURE_SEARCH_KEY', 'XXX')

#Once you've set key, you can use search_images_bing. This function is provided by the small utils class included with the notebooks online. If you're not sure where a function is defined, you can just type it in your notebook to find out:
search_images_bing

results = search_images_bing(key, 'grizzly bear')
ims = results.attrgot('content_url')
len(ims)

HTTPError                                 Traceback (most recent call last)
<ipython-input-18-cddb73f3292e> in <module>()
----> 1 results = search_images_bing(key, 'grizzly bear')
      2 ims = results.attrgot('content_url')
      3 len(ims)
1 frames
/usr/local/lib/python3.6/dist-packages/requests/models.py in raise_for_status(self)
    939
    940         if http_error_msg:
--> 941             raise HTTPError(http_error_msg, response=self)
    942
    943     def close(self):
HTTPError: 401 Client Error: PermissionDenied for url:
https://api.bing.microsoft.com/v7.0/images/search?
q=grizzly+bear&count=150&min_height=128&min_width=128

I am using Google Colab.
Maybe I have to add that I don't know what the key name is that I should replace the 'XXX' with. Jeremy Howard said that you would get one when signing up to Azure, but I didn't.


title: Fastai v2 IndexError: Target 2 is out of bounds

body:
Package version and reference page
Hi, I am referring to the page Blurr to combining the HuggingFace Transformers' models with the Fastai fine-tuning policies. Below are the versions of the packages:

transformers : 4.2.2
fastai : 2.1.10
ohmeow-blurr  : 0.0.22

The difference between the sample code in Blurr and mine is that I am trying to solve the multi-class single label text classification problem with 4 class of total labels while the tutorial is the binary-classification issue.
The dataset for both training and validation has 3 columns: texts, label, is_valid.

texts contains text data similar to the Blurr.
label contains 4 classes of string labels, for each raw, there is only one label.
is_valid contains bool value "True and False" that indicates if these raw are for training or validation.

Error message
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-16-fc93db149c6f> in <module>
     13 learn.freeze()
     14
---> 15 learn.fit_one_cycle(3, lr_max=1e-3)

~/fastai/fastai/callback/schedule.py in fit_one_cycle(self, n_epoch, lr_max, div, div_final, pct_start, wd, moms, cbs, reset_opt)
    110     scheds = {'lr': combined_cos(pct_start, lr_max/div, lr_max, lr_max/div_final),
    111               'mom': combined_cos(pct_start, *(self.moms if moms is None else moms))}
--> 112     self.fit(n_epoch, cbs=ParamScheduler(scheds)+L(cbs), reset_opt=reset_opt, wd=wd)
    113
    114 # Cell

~/fastai/fastai/learner.py in fit(self, n_epoch, lr, wd, cbs, reset_opt)
    204             self.opt.set_hypers(lr=self.lr if lr is None else lr)
    205             self.n_epoch = n_epoch
--> 206             self._with_events(self._do_fit, 'fit', CancelFitException, self._end_cleanup)
    207
    208     def _end_cleanup(self): self.dl,self.xb,self.yb,self.pred,self.loss = None,(None,),(None,),None,None

~/fastai/fastai/learner.py in _with_events(self, f, event_type, ex, final)
    153
    154     def _with_events(self, f, event_type, ex, final=noop):
--> 155         try:       self(f'before_{event_type}')       ;f()
    156         except ex: self(f'after_cancel_{event_type}')
    157         finally:   self(f'after_{event_type}')        ;final()

~/fastai/fastai/learner.py in _do_fit(self)
    195         for epoch in range(self.n_epoch):
    196             self.epoch=epoch
--> 197             self._with_events(self._do_epoch, 'epoch', CancelEpochException)
    198
    199     def fit(self, n_epoch, lr=None, wd=None, cbs=None, reset_opt=False):

~/fastai/fastai/learner.py in _with_events(self, f, event_type, ex, final)
    153
    154     def _with_events(self, f, event_type, ex, final=noop):
--> 155         try:       self(f'before_{event_type}')       ;f()
    156         except ex: self(f'after_cancel_{event_type}')
    157         finally:   self(f'after_{event_type}')        ;final()

~/fastai/fastai/learner.py in _do_epoch(self)
    189
    190     def _do_epoch(self):
--> 191         self._do_epoch_train()
    192         self._do_epoch_validate()
    193

~/fastai/fastai/learner.py in _do_epoch_train(self)
    181     def _do_epoch_train(self):
    182         self.dl = self.dls.train
--> 183         self._with_events(self.all_batches, 'train', CancelTrainException)
    184
    185     def _do_epoch_validate(self, ds_idx=1, dl=None):

~/fastai/fastai/learner.py in _with_events(self, f, event_type, ex, final)
    153
    154     def _with_events(self, f, event_type, ex, final=noop):
--> 155         try:       self(f'before_{event_type}')       ;f()
    156         except ex: self(f'after_cancel_{event_type}')
    157         finally:   self(f'after_{event_type}')        ;final()

~/fastai/fastai/learner.py in all_batches(self)
    159     def all_batches(self):
    160         self.n_iter = len(self.dl)
--> 161         for o in enumerate(self.dl): self.one_batch(*o)
    162
    163     def _do_one_batch(self):

~/fastai/fastai/learner.py in one_batch(self, i, b)
    177         self.iter = i
    178         self._split(b)
--> 179         self._with_events(self._do_one_batch, 'batch', CancelBatchException)
    180
    181     def _do_epoch_train(self):

~/fastai/fastai/learner.py in _with_events(self, f, event_type, ex, final)
    153
    154     def _with_events(self, f, event_type, ex, final=noop):
--> 155         try:       self(f'before_{event_type}')       ;f()
    156         except ex: self(f'after_cancel_{event_type}')
    157         finally:   self(f'after_{event_type}')        ;final()

~/fastai/fastai/learner.py in _do_one_batch(self)
    164         self.pred = self.model(*self.xb)
    165         self('after_pred')
--> 166         if len(self.yb): self.loss = self.loss_func(self.pred, *self.yb)
    167         self('after_loss')
    168         if not self.training or not len(self.yb): return

~/fastai/fastai/losses.py in __call__(self, inp, targ, **kwargs)
     31         if targ.dtype in [torch.int8, torch.int16, torch.int32]: targ = targ.long()
     32         if self.flatten: inp = inp.view(-1,inp.shape[-1]) if self.is_2d else inp.view(-1)
---> 33         return self.func.__call__(inp, targ.view(-1) if self.flatten else targ, **kwargs)
     34
     35 # Cell

~/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--> 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

~/anaconda3/lib/python3.8/site-packages/torch/nn/modules/loss.py in forward(self, input, target)
    959
    960     def forward(self, input: Tensor, target: Tensor) -> Tensor:
--> 961         return F.cross_entropy(input, target, weight=self.weight,
    962                                ignore_index=self.ignore_index, reduction=self.reduction)
    963

~/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   2460         tens_ops = (input, target)
   2461         if any([type(t) is not Tensor for t in tens_ops]) and has_torch_function(tens_ops):
-> 2462             return handle_torch_function(
   2463                 cross_entropy, tens_ops, input, target, weight=weight,
   2464                 size_average=size_average, ignore_index=ignore_index, reduce=reduce,

~/anaconda3/lib/python3.8/site-packages/torch/overrides.py in handle_torch_function(public_api, relevant_args, *args, **kwargs)
   1058         # Use `public_api` instead of `implementation` so __torch_function__
   1059         # implementations can do equality/identity comparisons.
-> 1060         result = overloaded_arg.__torch_function__(public_api, types, args, kwargs)
   1061
   1062         if result is not NotImplemented:

~/fastai/fastai/torch_core.py in __torch_function__(self, func, types, args, kwargs)
    317 #         if func.__name__[0]!='_': print(func, types, args, kwargs)
    318 #         with torch._C.DisableTorchFunction(): ret = _convert(func(*args, **(kwargs or {})), self.__class__)
--> 319         ret = super().__torch_function__(func, types, args=args, kwargs=kwargs)
    320         if isinstance(ret, TensorBase): ret.set_meta(self, as_copy=True)
    321         return ret

~/anaconda3/lib/python3.8/site-packages/torch/tensor.py in __torch_function__(cls, func, types, args, kwargs)
    993
    994         with _C.DisableTorchFunction():
--> 995             ret = func(*args, **kwargs)
    996             return _convert(ret, cls)
    997

~/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   2466     if size_average is not None or reduce is not None:
   2467         reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 2468     return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
   2469
   2470

~/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
   2262                          .format(input.size(0), target.size(0)))
   2263     if dim == 2:
-> 2264         ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
   2265     elif dim == 4:
   2266         ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

IndexError: Target 2 is out of bounds.

Code
While in the learn.fit_one_cycle(3, lr_max=1e-3) it shows the error message above. I am sure that for both the training set and the validation set have the same labels by checking len(df[df['is_valid']==False]['label'].unique()) == len(df[df['is_valid']==True]['label'].unique()) and print them respectively.
import os
import torch
from transformers import *
from fastai.text.all import *

from blurr.data.all import *
from blurr.modeling.all import *

task = HF_TASKS_AUTO.SequenceClassification
pretrained_model_name = "bert-base-uncased"
hf_arch, hf_config, hf_tokenizer, hf_model = BLURR_MODEL_HELPER.get_hf_objects(pretrained_model_name,  task=task)

# data loader
blocks = (HF_TextBlock(hf_arch, hf_config, hf_tokenizer, hf_model),  CategoryBlock)
dblock = DataBlock(blocks=blocks,
                   get_x=ColReader('texts'),
                   get_y=ColReader('label'),
                   splitter=ColSplitter('is_valid'))

dls = dblock.dataloaders(df, batch_size=4)

model = HF_BaseModelWrapper(hf_model)

learn = Learner(dls,
                model,
                opt_func=partial(Adam, decouple_wd=True),
                loss_func=CrossEntropyLossFlat(),
                metrics=[accuracy, F1Score(average='macro')],
                cbs=[HF_BaseModelCallback],
                splitter=hf_splitter)

learn.create_opt()
learn.freeze()

learn.fit_one_cycle(3, lr_max=1e-3)

Has anyone gotten some similar problems before? and is there anything I lost in the code?


title: How can i use pytorch model in C#?

body:
I have a pytorch model in NLP and a script for use it in python. Now i want to use this script in C#. I tried run python script from C# and it worked. I get user sentence in C#, pass it to python and its outputs use in C#. The problem is that i want to do this work in a loop until user select exit but every time it goes to python code, it must load pytorch model and its time consuming.
Is there a way to load model one time and then in a loop get input from user and inference with loaded model?


title: pytorch with GPU, CUBLAS_STATUS_EXECUTION_FAILED error, how to trace or fix it?

body:
I'm a newbie to GPU training and pytorch.
While executing a NMT code, got CUBLAS_STATUS_EXECUTION_FAILED error. (of course, with CPU, It performed well..)  MY GPU is not so excellent, I know, but the error is occuring every time with 1 batch size.
Please help me to fix or trace this.
Epoch [1/30]:   1%|▍                                                                    Epoch [1/30]:   1%|                 | 1/166 [00:00<?, ?it/s, loss=7.72, ppl=2.24e+3, |g_Epoch [1/30]:   1%|                 | 1/166 [00:00<?, ?it/s, loss=7.72, ppl=2.24e+3, |g_Epoch [1/30]:   1%|                 | 1/166 [00:00<?, ?it/s, loss=7.72, ppl=2.25e+3, |g_Epoch [1/30]:   1%|         | 2/166 [00:00<00:22,  7.43it/s, loss=7.72, ppl=2.25e+3, |g_Epoch [1/30]:   1%|         | 2/166 [00:00<00:22,  7.39it/s, loss=7.72, ppl=2.25e+3, |g_Epoch [1/30]:   2%|▏        | 3/166 [00:00<00:11, 14.74it/s, loss=7.72, ppl=2.25e+3, |g_Epoch [1/30]:   2%|▏        | 3/166 [00:00<00:11, 14.74it/s, loss=7.72, ppl=2.25e+3, |g_Epoch [1/30]:   2%|▏        | 3/166 [00:00<00:11, 14.74it/s, loss=7.72, ppl=2.26e+3, |g_Epoch [1/30]:   2%|▏        | 4/166 [00:00<00:10, 14.74it/s, loss=7.72, ppl=2.26e+3, |g_Epoch [1/30]:   2%|▏        | 4/166 [00:00<00:10, 14.74it/s, loss=7.72, ppl=2.26e+3, |g_Epoch [1/30]:   3%|▎        | 5/166 [00:00<00:13, 12.01it/s, loss=7.72, ppl=2.26e+3, |g_param|=4.27e+5, |param|=1.29e+3]

Current run is terminating due to exception: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasGemmEx( handle, opa, opb, m, n, k, &falpha, a, CUDA_R_16F, lda, b, CUDA_R_16F, ldb, &fbeta, c, CUDA_R_16F, ldc, CUDA_R_32F, CUBLAS_GEMM_DFALT_TENSOR_OP)`

 File "/home/00/pythonp/python38/pytorch/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 94, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/00/pythonp/python38/pytorch/lib/python3.8/site-packages/torch/nn/functional.py", line 1753, in linear
    return torch._C._nn.linear(input, weight, bias)
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasGemmEx( handle, opa, opb, m, n, k, &falpha, a, CUDA_R_16F, lda, b, CUDA_R_16F, ldb, &fbeta, c, CUDA_R_16F, ldc, CUDA_R_32F, CUBLAS_GEMM_DFALT_TENSOR_OP)`

My GPU is..
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro P620         On   | 00000000:81:00.0 Off |                  N/A |
| 34%   42C    P8    N/A /  N/A |     19MiB /  1999MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      2662      G   /usr/lib/xorg/Xorg                  9MiB |
|    0   N/A  N/A      3026      G   /usr/bin/gnome-shell                4MiB |
+-----------------------------------------------------------------------------+

With 'CUBLAS_STATUS_EXECUTION_FAILED' keyword, there're many advices to upgrade  CUDA Version to 11.2. However, It's already 11.2.
GPU Memory Usage..
| 34%   41C    P8    N/A /  N/A |     19MiB /  1999MiB |      0%      Default |
| 34%   41C    P0    N/A /  N/A |     64MiB /  1999MiB |      2%      Default |
| 34%   42C    P0    N/A /  N/A |    202MiB /  1999MiB |      4%      Default |
| 34%   43C    P0    N/A /  N/A |    346MiB /  1999MiB |      3%      Default |
| 34%   43C    P0    N/A /  N/A |    498MiB /  1999MiB |      3%      Default |
| 34%   43C    P0    N/A /  N/A |    632MiB /  1999MiB |      3%      Default |
| 34%   43C    P0    N/A /  N/A |    850MiB /  1999MiB |      2%      Default |
| 34%   44C    P0    N/A /  N/A |    962MiB /  1999MiB |     47%      Default |
| 34%   44C    P0    N/A /  N/A |    962MiB /  1999MiB |     45%      Default |
| 34%   45C    P0    N/A /  N/A |     19MiB /  1999MiB |     19%      Default |

...


title: How do I get words from an embedded vector?

body:
How can I convert them into their original words when I generate word vectors in the generator?
I used the nn.Embedding module built into pytorch to embed words.


title: Facing RuntimeError: invalid argument 3: sizes do not match

body:
My loss function is NLL loss,which takes in as inputs shape as  [108416, 3] and targets shape as [108416] and i get a resulting loss value of 2.2623 , but after the loss computation when i do the optimizer.step() call. I get

AND THIS IS LOSS Variable containing:
 2.2623
[torch.cuda.FloatTensor of size 1 (GPU 0)]

Traceback (most recent call last):
  File "/mnt/sdc1/project/training/fpr4x_liver_1x_2channel.py", line 336, in <module>
train_fpr4x_liver_1x_2channel_model()
  File "/mnt/sdc1/project/training/fpr4x_liver_1x_2channel.py", line 245, in train_fpr4x_liver_1x_2channel_model
est.run_experiment(opts.num_epochs, 5000,50)
  File "/media/redible/sdc/project/training/expt_utils.py", line 236, in run_experiment
self.trainer.train()
  File "/media/redible/sdc/project/training/expt_utils.py", line 75, in train
loss, outputs = self.net_mgr._forward_backward(network_inputs, loss_inputs)
  File "/media/redible/sdc/project/training/network_manager.py", line 19, in _forward_backward
self.optimizer.step()
  File "/usr/local/lib/python3.5/dist-packages/torch/optim/adam.py", line 69, in step
exp_avg.mul_(beta1).add_(1 - beta1, grad)
RuntimeError: invalid argument 3: sizes do not match at /pytorch/torch/lib/THC/generated/../generic/THCTensorMathPointwise.cu:271

Not sure whats the cause of the error any help would be appreciated,Thanks in advance.


title: How to reproduce a linear regression done via pseudo inverse in pytorch

body:
I try to reproduce a simple linear regression x = A†b using pytorch. But I get completely different numbers.

So first I use plain numpy and do

A_pinv = np.linalg.pinv(A)
betas = A_pinv.dot(b)

print(((b - A.dot(betas))**2).mean())
print(betas)

which results in:

364.12875
[0.43196774 0.14436531 0.42414093]

Now I try to get similar enough numbers using pytorch:

# re-implement via pytoch model using built-ins
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import TensorDataset, DataLoader

# We'll create a TensorDataset, which allows access to rows from inputs and targets as tuples.
# We'll also create a DataLoader, to split the data into batches while training.
# It also provides other utilities like shuffling and sampling.

inputs = to.from_numpy(A)
targets = to.from_numpy(b)
train_ds = TensorDataset(inputs, targets)

batch_size = 5
train_dl = DataLoader(train_ds, batch_size, shuffle=True)

# define model, loss and optimizer
new_model = nn.Linear(source_variables, predict_variables, bias=False)
loss_fn = F.mse_loss
opt = to.optim.SGD(new_model.parameters(), lr=1e-10)

def fit(num_epochs, new_model, loss_fn, opt):
    for epoch in tnrange(num_epochs, desc="epoch"):
        for xb,yb in train_dl:
            # Generate predictions
            pred = new_model(xb)
            loss = loss_fn(pred, yb)
            # Perform gradient descent
            loss.backward()
            opt.step()
            opt.zero_grad()

        if epoch % 1000 == 0:
            print((new_model.weight, loss))
    print('Training loss: ', loss_fn(model(inputs), targets))

# fit the model
fit(10000, new_model, loss_fn, opt)

It prints as the last result:

tensor([[0.0231, 0.5185, 0.4589]], requires_grad=True), tensor(271.8525, grad_fn=<MseLossBackward>))
Training loss:  tensor(378.2871, grad_fn=<MseLossBackward>)

As you can see these numbers are completely different so I must have made a mistake somewhere ...

Here are the numbers for A and b to reproduce the result:

A = np.array([[2822.48, 2808.48, 2810.92],
             [2832.94, 2822.48, 2808.48],
             [2832.57, 2832.94, 2822.48],
             [2824.23, 2832.57, 2832.94],
             [2854.88, 2824.23, 2832.57],
             [2800.71, 2854.88, 2824.23],
             [2798.36, 2800.71, 2854.88],
             [2818.46, 2798.36, 2800.71],
             [2805.37, 2818.46, 2798.36],
             [2815.44, 2805.37, 2818.46]], dtype=float32)

b = np.array([2832.94, 2832.57, 2824.23, 2854.88, 2800.71, 2798.36, 2818.46, 2805.37, 2815.44, 2834.4 ], dtype=float32)


title: How do I change a torch tensor to concat with another tensor

body:
I'm trying to concatenate a tensor of numerical data with the output tensor of a resnet-50 model.  The output of that model is tensor shape torch.Size([10,1000]) and the numerical data is tensor shape torch.Size([10, 110528,8]) where the 10 is the batch size, 110528 is the number of observations in a data frame sense, and 8 is the number of columns (in a dataframe sense).  I need to reshape the numerical tensor to torch.Size([10,8]) so it will concatenate properly.

How would I reshape the tensor?


title: PyTorch training attribute doesn't exists in C++ module

body:
I have a script that was traced and saved using torch.jit.save, when I load it in python using:
  net = torch.jit.load(r"script.pt")
  net.to('cuda')
  net.train()
  print(net.training)
  net.eval()
  print(net.training)

I get the expected output:
True
False

However, when I load the same file using libtorch with this code:
  auto module = torch::jit::load("script.pt");
  module.to(torch::kCUDA);
  module.train();
  std::cout << std::boolalpha << module.is_training() << std::endl;
  module.eval();
  std::cout << std::boolalpha << module.is_training() << std::endl;

the output is
true
true

stepping into the is_training() function, it looks for the training attribute but it doesn't exists so it defaults to true
  /// True if the module is in training mode.
  bool is_training() const {
    return attr("training", true).toBool();
  }

Using this module works, I can call its forward and process data, but I need to be able to set the training mode to false
Any thoughts? Is there something that needs to be done when tracing?
I'm using PyTorch 1.5.0 on Ubuntu 16.04 with CUDA 10.2


title: importing error default_collate is not defined

body:
I tried to import default_collate from torch.utils.data.dataloader but it gives me default_collate is not exsit and another function is exist (_collate_fn_t)
from torch.utils.data.dataloader import default_collate
from torch.utils.data.dataloader import _collate_fn_t
Are these commands are similar


title: Linear Regression with Pytorch - Prediction error

body:
I am using PyTorch to predict the value of a dependent variable.
The source file I am reading for dataset

As you can see that the Defect Per cent (which is a dependent variable is ~ 0 to 3)
import torch.nn as nn
import numpy as np
import torch
import pandas as pd
from torch.utils.data import TensorDataset, DataLoader
import torch.nn.functional as F

SourceData=pd.read_excel("Supplier Past Performance.xlsx") # Load the data into Pandas DataFrame

SourceData_train_independent= SourceData.drop(["Defect Per cent"], axis=1) # Drop depedent variable from training dataset

SourceData_train_dependent=SourceData["Defect Per cent"].copy() # Dependent variable value for training dataset

X_train = torch.tensor(SourceData_train_independent.values)
y_train=torch.tensor(SourceData_train_dependent.values)

X_train=X_train.type(torch.FloatTensor) #convert the type of tensor
y_train=y_train.type(torch.FloatTensor) #convert the type of tensor

# Define dataset
train_ds = TensorDataset(X_train, y_train)

# Define data loader
batch_size = 5
train_dl = DataLoader(train_ds, batch_size, shuffle=True)

#Define model
model = nn.Linear(3,1)

#Define optimizer
opt = torch.optim.SGD(model.parameters(), lr=0.02)

#Define loss function
loss_fn = F.mse_loss

#Define a utility function to train the model
def fit(num_epochs, model, loss_fn, opt):
    for epoch in range(num_epochs):
        for xb,yb in train_dl:

            #Generate predictions
            pred = model(xb)

            loss = loss_fn(pred,yb)
            #Perform gradient descent
            loss.backward()
            opt.step()
            opt.zero_grad()
   print('Training loss: ', loss_fn(model(xb), yb))

#Train the model for 100 epochs
fit(100, model, loss_fn, opt)

new_var=torch.Tensor([[5000.0, 33.0, 23.0]])
preds = model(new_var)
print(preds.item())

I am getting the prediction for the new_var as 963.40. This value is way higher that he expected value of 0.1 to 3.
Please help


title: Getting a TypeError when trying to save Optuna.Study in an optimize callback

body:
I'm running some Optuna study, and when I try to save it with joblib.dump, I'm getting the following:

TypeError: cannot pickle '_io.TextIOWrapper' object

I also try to pickle.dump(study, open('name.pkl', 'wb')), but I'm getting the same in both cases
EDIT: now I understood it has something to do with the fact I'm doing it in a callback, otherwise it works
I am training a custom PyTorch model, everything else seem to work in terms of the study itself.. but when I try to save it I get the above error.
I'm kind of clueless in terms of how to debug it.
Environment:

QubeOS with debian 10 VM
Python 3.8
PyTorch 1.5
Optuna 2.0
edited:
this is the stacktrace:

File "<ipython-input-3-f2187f915338>", line 1, in <module>
    joblib.dump(study, f'{filepath}.pkl')
  File "/opt/conda/envs/torch_38/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 480, in dump
    NumpyPickler(f, protocol=protocol).dump(value)
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 485, in dump
    self.save(obj)
  File "/opt/conda/envs/torch_38/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 282, in save
    return Pickler.save(self, obj)
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 601, in save
    self.save_reduce(obj=obj, *rv)
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 715, in save_reduce
    save(state)
  File "/opt/conda/envs/torch_38/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 282, in save
    return Pickler.save(self, obj)
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 558, in save
    f(self, obj)  # Call unbound method with explicit self
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 969, in save_dict
    self._batch_setitems(obj.items())
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 995, in _batch_setitems
    save(v)
  File "/opt/conda/envs/torch_38/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 282, in save
    return Pickler.save(self, obj)
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 601, in save
    self.save_reduce(obj=obj, *rv)
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 715, in save_reduce
    save(state)
  File "/opt/conda/envs/torch_38/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 282, in save
    return Pickler.save(self, obj)
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 558, in save
    f(self, obj)  # Call unbound method with explicit self
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 969, in save_dict
    self._batch_setitems(obj.items())
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 995, in _batch_setitems
    save(v)
  File "/opt/conda/envs/torch_38/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 282, in save
    return Pickler.save(self, obj)
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 601, in save
    self.save_reduce(obj=obj, *rv)
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 715, in save_reduce
    save(state)
  File "/opt/conda/envs/torch_38/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 282, in save
    return Pickler.save(self, obj)
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 558, in save
    f(self, obj)  # Call unbound method with explicit self
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 969, in save_dict
    self._batch_setitems(obj.items())
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 995, in _batch_setitems
    save(v)
  File "/opt/conda/envs/torch_38/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 282, in save
    return Pickler.save(self, obj)
  File "/opt/conda/envs/torch_38/lib/python3.8/pickle.py", line 576, in save
    rv = reduce(self.proto)
TypeError: cannot pickle '_io.TextIOWrapper' object


title: Gaussian 1D Curve in Pytorch

body:
I am writing a code that takes in a float standard deviation value and returns a kernel as a column vector.
However, when I run my code, it seems like the Gaussian curve is wrong. I know that Equation for 1D Gaussian curve is defined as:
 where I have defined kernel size k as 4*sigma+1 and mean value mu as k/2.
The wrong Gaussian curve and its tensor values are in
This is my code:
def create_1D_Gaussian_kernel(standard_deviation: float) -> torch.FloatTensor:
    """Creates a 1D Gaussian kernel using the specified standard deviation.

    Note: ensure that the value of the kernel sums to 1.

    Args:
        standard_deviation (float): standard deviation of the gaussian

    Returns:
        torch.FloatTensor: required kernel as a column vector
    """

    kernel = torch.FloatTensor()
    torch.pi = torch.acos(torch.zeros(1)).item() *2 # define pi from torch

    kernel_size_k = 4 * standard_deviation + 1
    kernel = torch.arange(kernel_size_k)
    mean_mu = kernel_size_k / 2
    variance = standard_deviation ** 2
    normalization_Z = 1 / (torch.sqrt(torch.tensor(2 * torch.pi)) * standard_deviation)
    kernel = normalization_Z * torch.exp(-((kernel - mean_mu) ** 2) / (2 * variance))

    return kernel


title: Use pre trained Nodes from past runs - Pytorch Biggraph

body:
After struggling with this amazing facebookresearch
/
PyTorch-BigGraph project, and its impossible API, I managed to get a grip on how to run it (thanks to stand alone simple example)
My system restrictions do not allow me to train the dense (embedding) representation of all edges, and I need from time to time to upload past embeddings and train the model using both new edges and existing nodes, notice that nodes in past and new edge list do not necessarily overlap.
I tried to understand from here: see the context section how to do it, so far with no success.
Following is a stand-alone PGD code, that turned batch_edges into an embedding node list, however, I need it to use pre-trained nodes list past_trained_nodes.
import os
import shutil
from pathlib import Path

from torchbiggraph.config import parse_config
from torchbiggraph.converters.importers import TSVEdgelistReader, convert_input_data
from torchbiggraph.train import train
from torchbiggraph.util import SubprocessInitializer, setup_logging

DIMENSION = 4
DATA_DIR = 'data'
GRAPH_PATH = DATA_DIR + '/output1.tsv'
MODEL_DIR = 'model'

raw_config = dict(
        entity_path=DATA_DIR,
        edge_paths=[DATA_DIR + '/edges_partitioned', ],
        checkpoint_path=MODEL_DIR,
        entities={"n": {"num_partitions": 1}},
        relations=[{"name": "doesnt_matter", "lhs": "n", "rhs": "n", "operator": "complex_diagonal", }],
        dynamic_relations=False, dimension=DIMENSION, global_emb=False, comparator="dot",
        num_epochs=7, num_uniform_negs=1000, loss_fn="softmax", lr=0.1, eval_fraction=0.,)

batch_edges = [["A", "B"], ["B", "C"],  ["C", "D"], ["D", "B"], ["B", "D"]]

# I want the model to use these pretrained nodes, Notice that Node A exist, And F Does not
#I dont have all past nodes, as some are gained from data
past_trained_nodes = {'A': [0.5, 0.3, 1.5, 8.1], 'F': [3, 0.6, 1.2, 4.3]}

try:
    shutil.rmtree('data')
except:
    pass
try:
    shutil.rmtree(MODEL_DIR)
except:
    pass

os.makedirs(DATA_DIR, exist_ok=True)
with open(GRAPH_PATH, 'w') as f:
    for edge in batch_edges:
        f.write('\t'.join(edge) + '\n')

setup_logging()
config = parse_config(raw_config)
subprocess_init = SubprocessInitializer()
input_edge_paths = [Path(GRAPH_PATH)]

convert_input_data(config.entities, config.relations, config.entity_path, config.edge_paths,
                   input_edge_paths, TSVEdgelistReader(lhs_col=0, rel_col=None, rhs_col=1),
                   dynamic_relations=config.dynamic_relations, )

train(config, subprocess_init=subprocess_init)

How can I use my pre-trained nodes in the current model?
Thanks in advance!


title: PyTorch get_device_capability() Output Explanation

body:
when running torch.cuda.get_device_capability() on my GTX 1070 I get the following output: (6, 1).
Could someone explain what this means?


title: How to create and use PyTorch learnable scalar variables outside of nn.Module?

body:
I am working on a multi-objective problem where I have multiple losses that I need to compute, and the total loss is just the sum of the losses. I want to have PyTorch learnable floating-point parameters alpha, and beta that act as coefficients to the individual losses. Note that the summation of losses occurs outside my NN model in the training loop:
optimizer = AdamW(model.parameters(), lr=2e-5, eps=1e-8)

for batch in dl:

    optimizer.zero_grad()

    result = model(batch)

    loss1 = loss_fn_1(result)
    loss2 = loss_fn_2(result)
    loss3 = loss_fn_3(result)

    loss = alpha*loss1 + beta*loss2 + (1-beta)*loss3 # How to optimize alpha, beta?

    loss.backward()
    optimizer.step()

How would I declare and use the learnable parameters alpha and beta?


title: Load trained model on another machine - fastai, torch, huggingface

body:
I am using fastai with pytorch to fine tune XLMRoberta from huggingface.
I've trained the model and everything is fine on the machine where I trained it.
But when I try to load the model on another machine I get OSError - Not Found - No such file or directory pointing to .cache/torch/transformers/. The issue is the path of a vocab_file.
I've used fastai's Learner.export to export the model in .pkl file, but I don't believe that issue is related to fastai since I found the same issue appearing in flairNLP.
It appears that the path to the cache folder, where the vocab_file is stored during the training, is embedded in the .pkl file:

The error comes from transformer's XLMRobertaTokenizer __setstate__:
def __setstate__(self, d):
    self.__dict__ = d
    self.sp_model = spm.SentencePieceProcessor()
    self.sp_model.Load(self.vocab_file)

which tries to load the vocab_file using the path from the file.
I've tried patching this method using:
pretrained_model_name = "xlm-roberta-base"
vocab_file = XLMRobertaTokenizer.from_pretrained(pretrained_model_name).vocab_file

def _setstate(self, d):
    self.__dict__ = d
    self.sp_model = spm.SentencePieceProcessor()
    self.sp_model.Load(vocab_file)

XLMRobertaTokenizer.__setstate__ = MethodType(_setstate, XLMRobertaTokenizer(vocab_file))

And that successfully loaded the model but caused other problems like missing model attributes and other unwanted issues.
Can someone please explain why is the path embedded inside the file, is there a way to configure it without reexporting the model or if it has to be reexported how to configure it dynamically using fastai, torch and huggingface.


title: Python Error: the following arguments are required: experimentname

body:
I am a beginner in Python and PyTorch environment. I am executing the program on GitHub.
I am getting the following error:
usage: main.py [-h] [--self_host SELF_HOST] [--cpu] [--port PORT]
               [--server_port SERVER_PORT] [--config CONFIG]
               [--devices_per_trial DEVICES_PER_TRIAL] [--dataroot DATAROOT]
               [--s3 S3] [--logroot LOGROOT] [--seed SEED]
               [--num_eigenthings NUM_EIGENTHINGS] [--batch_size BATCH_SIZE]
               [--eval_batch_size EVAL_BATCH_SIZE] [--momentum MOMENTUM]
               [--num_steps NUM_STEPS] [--max_samples MAX_SAMPLES] [--cuda]
               [--full_dataset] [--fname FNAME] [--mode {power_iter,lanczos}]
               experimentname
main.py: error: the following arguments are required: experimentname
An exception has occurred, use %tb to see the full traceback.

SystemExit: 2

Upon %tb, I get following details:
Traceback (most recent call last):

  File "/home/ajinkya/Hessian_Expts/main.py", line 77, in <module>
    skeletor.execute(main)

  File "/home/ajinkya/miniconda3/lib/python3.8/site-packages/skeletor/launcher.py", line 233, in execute
    args = _parser.val.parse_args()

  File "/home/ajinkya/miniconda3/lib/python3.8/argparse.py", line 1768, in parse_args
    args, argv = self.parse_known_args(args, namespace)

  File "/home/ajinkya/miniconda3/lib/python3.8/argparse.py", line 1800, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)

  File "/home/ajinkya/miniconda3/lib/python3.8/argparse.py", line 2034, in _parse_known_args
    self.error(_('the following arguments are required: %s') %

  File "/home/ajinkya/miniconda3/lib/python3.8/argparse.py", line 2521, in error
    self.exit(2, _('%(prog)s: error: %(message)s\n') % args)

  File "/home/ajinkya/miniconda3/lib/python3.8/argparse.py", line 2508, in exit
    _sys.exit(status)

SystemExit: 2

Kindly help to get rid of this error. Thanks in advance.


title: MMDetection loading from own training checkpoint for inference produces garbage detections

body:
I've trained up a very simple model using the MMDetection colab tutorial and then verifying the result using:
img = mmcv.imread('/content/mmdetection/20210301_145246_123456.jpg')
img = cv2.resize(img, (0,0), fx=0.25, fy=0.25)

model.cfg = cfg
result = inference_detector(model, img)
show_result_pyplot(model, img, result)

confirms that it's working great.
I then follow the same steps as for training but instead I load my own training checkpoint, and I don't train. Then running the verification snippet above produces garbage results.
Here's that in code
from mmcv import Config
cfg = Config.fromfile('configs/faster_rcnn/faster_rcnn_r50_caffe_fpn_mstrain_1x_coco.py')

from mmdet.apis import set_random_seed

# Modify dataset type and path
cfg.dataset_type = 'SamplesDataset'
cfg.data_root = 'samples_dataset/'

cfg.data.test.type = 'SamplesDataset'
cfg.data.test.data_root = 'samples_dataset/'
cfg.data.test.ann_file = 'train.txt'
cfg.data.test.img_prefix = 'o2h'

cfg.data.train.type = 'SamplesDataset'
cfg.data.train.data_root = 'samples_dataset/'
cfg.data.train.ann_file = 'train.txt'
cfg.data.train.img_prefix = 'o2h'

cfg.data.val.type = 'SamplesDataset'
cfg.data.val.data_root = 'samples_dataset/'
cfg.data.val.ann_file = 'val.txt'
cfg.data.val.img_prefix = 'o2h'

# modify num classes of the model in box head
cfg.model.roi_head.bbox_head.num_classes = 1
# We can still use the pre-trained Mask RCNN model though we do not need to
# use the mask branch
# cfg.load_from = 'checkpoints/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth'
cfg.load_from = './experiments/epoch_1.pth'

# Set up working dir to save files and logs.
cfg.work_dir = './experiments'

# The original learning rate (LR) is set for 8-GPU training.
# We divide it by 8 since we only use one GPU.
cfg.optimizer.lr = 0.02 / 8
cfg.lr_config.warmup = None
cfg.log_config.interval = 10
cfg.runner = dict(type='EpochBasedRunner', max_epochs=1)
cfg.total_epochs = 1

# Change the evaluation metric since we use customized dataset.
cfg.evaluation.metric = 'mAP'
# We can set the evaluation interval to reduce the evaluation times
# cfg.evaluation.interval = 12
# We can set the checkpoint saving interval to reduce the storage cost
cfg.checkpoint_config.interval = 1

# Set seed thus the results are more reproducible
cfg.seed = 0
set_random_seed(0, deterministic=False)
cfg.gpu_ids = range(1)

# We can initialize the logger for training and have a look
# at the final config used for training
# print(f'Config:\n{cfg.pretty_text}')

from mmdet.datasets import build_dataset
from mmdet.models import build_detector
from mmdet.apis import train_detector

# Build dataset
# datasets = [build_dataset(cfg.data.train)]

# Build the detector
model = build_detector(cfg.model)
# Add an attribute for visualization convenience
model.CLASSES = datasets[0].CLASSES

# Create work_dir
# mmcv.mkdir_or_exist(osp.abspath(cfg.work_dir))
# train_detector(model, datasets, cfg, distributed=False, validate=True)

Obviously, I wouldn't normally do all that just for validating my model, but this is one of many debugging steps for me, as my goal is to download and run the model locally. This is what I'm trying to do locally:
import sys
import glob
import time

sys.path.insert(0, '../mmdetection')
from mmdet.apis import init_detector, inference_detector, show_result_pyplot
from mmdet.models import build_detector
import mmcv
import numpy as np

file_paths = glob.glob('samples/o2h/*.jpg')

cfg = mmcv.Config.fromfile('../mmdetection/configs/faster_rcnn/faster_rcnn_r50_caffe_fpn_mstrain_1x_coco.py')
cfg.model.roi_head.bbox_head.num_classes = 1
cfg.load_from = 'models/mmdet_faster_rcnn_r50_caffe_fpn_mstrain_1x_coco.pth' # my own checkpoint
model = build_detector(cfg.model)
model.CLASSES = ('hash',)
model.cfg = cfg

file_path = np.random.choice(file_paths)
print(file_path)

start = time.time()
result = inference_detector(model, file_path)
print(f"Time taken for inference: {time.time() - start:.2f}s")
show_result_pyplot(model, file_path, result)


title: Why does my pytorch distributed training (DDP) code send a SIGKILL signal on its own?

body:
My code runs for a few interactions but before ending the training it sends a SIGKILL for some unknown reason:
backend='nccl'
rank=1
mp.current_process()=<SpawnProcess name='SpawnProcess-2' parent=13950 started>
os.getpid()=13987
setting up rank=1 (with world_size=4)
MASTER_ADDR='127.0.0.1'
44109
backend='nccl'
--> done setting up rank=0
--> done setting up rank=2
--> done setting up rank=1
--> done setting up rank=3
setup process done for rank=0
setup process done for rank=2
setup process done for rank=1
setup process done for rank=3
Starting training...

n_epoch=0
Traceback (most recent call last):
  File "/home/miranda9/ML4Coq/ml4coq-proj/embeddings_zoo/tree_nns/main_brando.py", line 279, in <module>
    main_distributed()
  File "/home/miranda9/ML4Coq/ml4coq-proj/embeddings_zoo/tree_nns/main_brando.py", line 188, in main_distributed
    spawn_return = mp.spawn(fn=train, args=(opts,), nprocs=opts.world_size)
  File "/home/miranda9/miniconda3/envs/automl-meta-learning/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 199, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/home/miranda9/miniconda3/envs/automl-meta-learning/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 157, in start_processes
    while not context.join():
  File "/home/miranda9/miniconda3/envs/automl-meta-learning/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 105, in join
    raise Exception(
Exception: process 1 terminated with signal SIGKILL

I don't understand why it does that. I am not incrementally storing anything as training goes so I don't think it should be a memory issue (especially if it trains fine for a few batches.
How do I even start debugging this with the error not giving me any information? Ideas?

In my research I've checked these links but none seem to help:

How does one fix a `Exception: process 0 terminated with signal SIGSEGV` error and if the single gpu code works fine?
https://github.com/huggingface/transformers/issues/3660
https://discuss.pytorch.org/t/exception-process-0-terminated-with-signal-sigkill/75570/5
https://github.com/PyTorchLightning/pytorch-lightning/issues/1590
Python script terminated by SIGKILL rather than throwing MemoryError
https://discuss.pytorch.org/t/torch-utils-data-dataloader-issue/92770
https://www.reddit.com/r/pytorch/comments/mdsljr/why_does_my_pytorch_distributed_training_ddp_code/
https://www.quora.com/unanswered/Why-does-my-PyTorch-distributed-training-DDP-code-send-a-SIGKILL-signal-on-its-own
https://github.com/pytorch/pytorch/issues/54823


title: How to resolve the error "mat1 and mat2 shapes cannot be multiplied" for the following CNN architecture?

body:
I am trying to implement a Conv1d model with Batch Normalization but I am getting the error :
RuntimeError                              Traceback (most recent call last)
<ipython-input-117-ef6e122ea50c> in <module>()
----> 1 test()
      2 for epoch in range(1, n_epochs + 1):
      3   train(epoch)
      4   test()

7 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in linear(input, weight, bias)
   1751     if has_torch_function_variadic(input, weight):
   1752         return handle_torch_function(linear, (input, weight), input, weight, bias=bias)
-> 1753     return torch._C._nn.linear(input, weight, bias)
   1754
   1755

RuntimeError: mat1 and mat2 shapes cannot be multiplied (32x140 and 100x10)

I use a batch size of 32, with the number of features of the data being 40. I've been trying to calculate where the 32 x 140 is coming from but I wasn't able to do that. Here is the architecture for the CNN I am trying to use:
class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        #self.flatten=nn.Flatten()
        self.net_stack=nn.Sequential(
            nn.Conv1d(in_channels=1, out_channels=25, kernel_size=5, stride=2), #applying batch norm
            nn.ReLU(),
            nn.BatchNorm1d(25, affine=True),
            nn.Conv1d(in_channels=25, out_channels=20, kernel_size=5, stride=2), #applying batch norm
            nn.ReLU(),
            nn.BatchNorm1d(20, affine=True),
            nn.Flatten()
            nn.Linear(20*5, 10),
            nn.Softmax(dim=1))

    def forward(self,x):
        # result=self.net_stack(x[None])
        result=self.net_stack(x[:, None, :])
        return result


title: Training segmentation model, 4 GPUs are working, 1 fills and getting: "CUDA error: out of memory"

body:
I'm trying to build a segmentation model,and I keep getting
 "CUDA error: out of memory",after ivestigating, I realized that all the 4 GPUs are working but one of them is filling.

Some technical details:

My Model:

the model is written in pytorch and has 3.8M parameters.
My Hardware:

I have 4 GPUs with 12GRAM (Titan V) each.

I'm trying to understand why one of my GPUs is filling up, and what am I doing wrong.

Evidence:
as can be seen from the screenshot below, all the GPUs are working, but one of them just keep filling until he gets his limit.

Code:
I'll try to explain what I did in the code:

First my model:

model = model.cuda()
model = nn.DataParallel(model, device_ids=None)

Second, Inputs and targets:

inputs = inputs.to('cuda')
masks = masks.to('cuda')

Those are the lines that working with the GPUs, if I missed something, and you need anything else, please share.

I'm feeling like I'm missing something so basic, that will affect not only this model but also the models in the future, I'll be more than happy for some help.

Thanks a lot!


title: TypeError - Torch.Device RepeatedScalarContainer

body:
I keep getting the error TypeError: new() received an invalid combination of arguments - got (google.protobuf.pyext._message.RepeatedScalarContainer, int).

I've printed the type of device, finding that it's of class torch.device and tested all of the relevant commands in a separate program. The relevant code is below and the error is below that. I'm using this along with Unity's ML-Agents. What could the problem be?

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print(type(device))

class Agent():
    """Interacts with and learns from the environment."""

    def __init__(self, state_size, action_size, random_seed):
        """Initialize an Agent object.

        Params
        ======
            state_size (int): dimension of each state
            action_size (int): dimension of each action
            random_seed (int): random seed
        """
        self.state_size = state_size
        self.action_size = action_size
        self.seed = random.seed(random_seed)

        # Actor Network (w/ Target Network)
        self.actor_local = Actor(state_size, action_size, random_seed).to(device)
        self.actor_target = Actor(state_size, action_size, random_seed).to(device)
        self.actor_optimizer = optim.Adam(self.actor_local.parameters(), lr=LR_ACTOR)

Traceback (most recent call last):
  File "train.py", line 87, in <module>
    agent_1 = Agent(state_size=48, action_size=action_size, random_seed=0)
  File "C:\Users\Tester\ml-agents\ml-agents\mlagents\trainers\ddpg\ddpg_agent.py", line 39, in __init__
    self.actor_local = Actor(state_size, action_size, random_seed).to(device)
  File "C:\Users\Tester\ml-agents\ml-agents\mlagents\trainers\ddpg\model.py", line 29, in __init__
    self.fc3 = nn.Linear(fc2_units, action_size)
  File "C:\Users\Tester\AppData\Local\conda\conda\envs\ml-agents\lib\site-packages\torch\nn\modules\linear.py", line 51, in __init__
    self.weight = Parameter(torch.Tensor(out_features, in_features))
TypeError: new() received an invalid combination of arguments - got (google.protobuf.pyext._message.RepeatedScalarContainer, int), but expected one of:
 * (torch.device device)
 * (torch.Storage storage)
 * (Tensor other)
 * (tuple of ints size, torch.device device)
      didn't match because some of the arguments have invalid types: (e[31;1mgoogle.protobuf.pyext._message.RepeatedScalarContainere[0m, e[31;1minte[0m)
 * (object data, torch.device device)
      didn't match because some of the arguments have invalid types: (e[31;1mgoogle.protobuf.pyext._message.RepeatedScalarContainere[0m, e[31;1minte[0m)


title: DDPG not converging for VRep maze environment

body:
I'm sorry for the lengthy post, just wanted to provide with implementation details in advance.

Also, sorry for the code, I know it's a mess.

I've been using a combination of PyTorch and VRep to setup environment for the differential drive robot to learn maneuvering the maze. The robot only has 6 IC proximity sensors along its rim that measure distances from 10-80cm. As a first simple version, I've created VRep environment containing just a narrow corridor (width of the corridor is ~2*robot_width). Starting position for the robot is somewhere in the middle of the corridor, with the goal point being somewhere at the end of the corridor, with goal velocities being ~0 m/s. Idea is that my learned agent handles both navigation and low level robot control at the same time.

Now, I've been using preexisting DDPG implementation which works on Gym environments, but it does not seem to converge for my case. I thought that using only 6 proximity sensors is the part of the problem, so I've introduced to the state vector position and velocity readings, together with errors for the desired state (as some papers suggest).

I would appreciate any help.

ddpg_agent

class Agent():
    """Interacts with and learns from the environment."""

    def __init__(self, state_size, action_size, random_seed):
        """Initialize an Agent object.

        Params
        ======
            state_size (int): dimension of each state
            action_size (int): dimension of each action
            random_seed (int): random seed
        """
        self.state_size = state_size
        self.action_size = action_size
        self.seed = random.seed(random_seed)

        # Actor Network (w/ Target Network)
        self.actor_local = Actor(state_size, action_size, random_seed).to(device)
        self.actor_target = Actor(state_size, action_size, random_seed).to(device)
        self.actor_optimizer = optim.Adam(self.actor_local.parameters(), lr=LR_ACTOR)

        # Critic Network (w/ Target Network)
        self.critic_local = Critic(state_size, action_size, random_seed).to(device)
        self.critic_target = Critic(state_size, action_size, random_seed).to(device)
        self.critic_optimizer = optim.Adam(self.critic_local.parameters(), lr=LR_CRITIC, weight_decay=WEIGHT_DECAY)

        # Noise process
        self.noise = OUNoise(action_size, random_seed)

        # Replay memory
        self.memory = ReplayBuffer(action_size, BUFFER_SIZE, BATCH_SIZE, random_seed)

    def step(self, state, action, reward, next_state, done):
        """Save experience in replay memory, and use random sample from buffer to learn."""
        # Save experience / reward
        self.memory.add(state, action, reward, next_state, done)

    def act(self, state, add_noise=True):
        """Returns actions for given state as per current policy."""
        state = torch.from_numpy(state).float().to(device)
        self.actor_local.eval()
        with torch.no_grad():
            action = self.actor_local(state).cpu().data.numpy()
        self.actor_local.train()
        if add_noise:
            action += self.noise.sample()
        return np.clip(action, -2, 2)

    def reset(self):
        self.noise.reset()

    def start_learn(self):
        if len(self.memory) > MIN_BUFFER_SIZE:
            experiences = self.memory.sample()
            self.learn(experiences, GAMMA)

    def learn(self, experiences, gamma):
        """Update policy and value parameters using given batch of experience tuples.
        Q_targets = r + γ * critic_target(next_state, actor_target(next_state))
        where:
            actor_target(state) -> action
            critic_target(state, action) -> Q-value

        Params
        ======
            experiences (Tuple[torch.Tensor]): tuple of (s, a, r, s', done) tuples
            gamma (float): discount factor
        """
        states, actions, rewards, next_states, dones = experiences

        # ---------------------------- update critic ---------------------------- #
        # Get predicted next-state actions and Q values from target models
        actions_next = self.actor_target(next_states)
        Q_targets_next = self.critic_target(next_states, actions_next)
        # Compute Q targets for current states (y_i)
        Q_targets = rewards + (gamma * Q_targets_next * (1 - dones)).detach()
        # Compute critic loss
        Q_expected = self.critic_local(states, actions)
        critic_loss = F.mse_loss(Q_expected, Q_targets)
        # Minimize the loss
        self.critic_optimizer.zero_grad()
        critic_loss.backward()
        # torch.nn.utils.clip_grad_norm_(self.critic_local.parameters(), 1)
        self.critic_optimizer.step()

        # ---------------------------- update actor ---------------------------- #
        # Compute actor loss
        actions_pred = self.actor_local(states)
        actor_loss = -self.critic_local(states, actions_pred).mean()
        # Minimize the loss
        self.actor_optimizer.zero_grad()
        actor_loss.backward()
        self.actor_optimizer.step()

        # ----------------------- update target networks ----------------------- #
        self.soft_update(self.critic_local, self.critic_target, TAU)
        self.soft_update(self.actor_local, self.actor_target, TAU)

    def soft_update(self, local_model, target_model, tau):
        """Soft update model parameters.
        θ_target = τ*θ_local + (1 - τ)*θ_target

        Params
        ======
            local_model: PyTorch model (weights will be copied from)
            target_model: PyTorch model (weights will be copied to)
            tau (float): interpolation parameter
        """
        for target_param, local_param in zip(target_model.parameters(), local_model.parameters()):
            target_param.data.copy_(tau*local_param.data + (1.0-tau)*target_param.data)

    def save_samples(self):
        fileSamples = open('samples.obj', 'w')
        pickle.dump(self.memory, fileSamples)

    def load_samples(self):
        fileSamples = open('samples.obj', 'r')
        return pickle.load(fileSamples)

class OUNoise:
    """Ornstein-Uhlenbeck process."""

    def __init__(self, size, seed, mu=0., theta=0.15, sigma=0.2):
        """Initialize parameters and noise process."""
        self.mu = mu * np.ones(size)
        self.theta = theta
        self.sigma = sigma
        self.seed = random.seed(seed)
        self.reset()

    def reset(self):
        """Reset the internal state (= noise) to mean (mu)."""
        self.state = copy.copy(self.mu)

    def sample(self):
        """Update internal state and return it as a noise sample."""
        x = self.state
        dx = self.theta * (self.mu - x) + self.sigma * np.array([np.random.randn() for i in range(len(x))])
        self.state = x + dx
        return self.state

class ReplayBuffer:
    """Fixed-size buffer to store experience tuples."""

    def __init__(self, action_size, buffer_size, batch_size, seed):
        """Initialize a ReplayBuffer object.
        Params
        ======
            buffer_size (int): maximum size of buffer
            batch_size (int): size of each training batch
        """
        self.action_size = action_size
        self.memory = deque(maxlen=buffer_size)  # internal memory (deque)
        self.batch_size = batch_size
        self.experience = namedtuple("Experience", field_names=["state", "action", "reward", "next_state", "done"])
        self.seed = random.seed(seed)

    def add(self, state, action, reward, next_state, done):
        """Add a new experience to memory."""
        e = self.experience(state, action, reward, next_state, done)
        self.memory.append(e)

    def sample(self):
        """Randomly sample a batch of experiences from memory."""
        experiences = random.sample(self.memory, k=self.batch_size)

        states = torch.from_numpy(np.vstack([e.state for e in experiences if e is not None])).float().to(device)
        actions = torch.from_numpy(np.vstack([e.action for e in experiences if e is not None])).float().to(device)
        rewards = torch.from_numpy(np.vstack([e.reward for e in experiences if e is not None])).float().to(device)
        next_states = torch.from_numpy(np.vstack([e.next_state for e in experiences if e is not None])).float().to(device)
        dones = torch.from_numpy(np.vstack([e.done for e in experiences if e is not None]).astype(np.uint8)).float().to(device)

        return (states, actions, rewards, next_states, dones)

    def __len__(self):
        """Return the current size of internal memory."""
        return len(self.memory)

Model:

class Actor(nn.Module):
    """Actor (Policy) Model."""

    def __init__(self, state_size, action_size, seed, fc1_units=600, fc2_units=400, fc3_units=300):
        """Initialize parameters and build model.
        Params
        ======
            state_size (int): Dimension of each state
            action_size (int): Dimension of each action
            seed (int): Random seed
            fc1_units (int): Number of nodes in first hidden layer
            fc2_units (int): Number of nodes in second hidden layer
        """
        super(Actor, self).__init__()
        self.seed = torch.manual_seed(seed)
        self.fc1 = nn.Linear(state_size, fc1_units)

        self.bn1 = nn.BatchNorm1d(fc1_units)

        self.fc2 = nn.Linear(fc1_units, fc2_units)
        self.fc3 = nn.Linear(fc2_units, fc3_units)
        self.fc4 = nn.Linear(fc3_units, action_size)
        self.reset_parameters()

    def reset_parameters(self):
        self.fc1.weight.data.uniform_(*hidden_init(self.fc1))
        self.fc2.weight.data.uniform_(*hidden_init(self.fc2))
        self.fc3.weight.data.uniform_(*hidden_init(self.fc3))
        self.fc4.weight.data.uniform_(-3e-3, 3e-3)

    def forward(self, state):
        """Build an actor (policy) network that maps states -> actions."""
        # x = F.relu(self.bn1(self.fc1(state.unsqueeze(0))))
        x = F.relu(self.fc1(state))
        x = F.relu(self.fc2(x))
        x = F.relu(self.fc3(x))
        return torch.tanh(self.fc4(x))

class Critic(nn.Module):
    """Critic (Value) Model."""

    def __init__(self, state_size, action_size, seed, fcs1_units=600, fc2_units=400, fc3_units=300):
        """Initialize parameters and build model.
        Params
        ======
            state_size (int): Dimension of each state
            action_size (int): Dimension of each action
            seed (int): Random seed
            fcs1_units (int): Number of nodes in the first hidden layer
            fc2_units (int): Number of nodes in the second hidden layer
        """
        super(Critic, self).__init__()
        self.seed = torch.manual_seed(seed)

        self.fcs1 = nn.Linear(state_size, fcs1_units)

        self.bn1 = nn.BatchNorm1d(fcs1_units)

        self.fc2 = nn.Linear(fcs1_units+action_size, fc2_units)
        self.fc3 = nn.Linear(fc2_units, fc3_units)
        self.fc4 = nn.Linear(fc3_units, 1)
        self.reset_parameters()

    def reset_parameters(self):
        self.fcs1.weight.data.uniform_(*hidden_init(self.fcs1))
        self.fc2.weight.data.uniform_(*hidden_init(self.fc2))
        self.fc3.weight.data.uniform_(*hidden_init(self.fc3))
        self.fc4.weight.data.uniform_(-3e-3, 3e-3)

    def forward(self, state, action):
        """Build a critic (value) network that maps (state, action) pairs -> Q-values."""
        # xs = F.relu(self.bn1(self.fcs1(state.unsqueeze(0))))
        xs = F.relu(self.fcs1(state))

        x = torch.cat((xs, action), dim=1)
        x = F.relu(self.fc2(x))
        x = F.relu(self.fc3(x))
        return self.fc4(x)

Main:

def main():
    vrepHeadlessMode = True

    state_dim = 18  #  x, y, yaw, vx, vy, v_yaw, e_x, e_y, e_yaw, e_vx, e_vy, e_v_yaw, prox 0 ... prox5
    action_dim = 2
    action_space = np.array([[-2, 2], [-2, 2]])
    action_lim = [-2.0, 2.0]  # 2 o/sec is the max angular speed of each motor, max. linear velocity is 0.5 m/s

    learn_every = 1  # number of steps after which the network update occurs [20]
    num_learn = 1  # number of network updates done in a row [10]

    episodes = 10000
    steps = 500

    desiredState = [-1.4, 0.3, -np.pi, 0.0, 0.0, 0.0]  # x, y, yawAngle, vx, vy, yawVelocity

    mobRob = MobRob(['MobRob'],
                    ['leftMotor', 'rightMotor'],
                    ['proximitySensor0', 'proximitySensor1', 'proximitySensor2', 'proximitySensor3', 'proximitySensor4',
                     'proximitySensor5'])
    env = LabEnv(mobRob, vrepHeadlessMode)

    random_seed = 7
    mobRob = Agent(state_dim, action_dim, random_seed)

    total_num_of_steps = 0
    actions = np.zeros((episodes, steps+1, action_dim), dtype=np.float)
    total_rewards = []
    save_rewards = []
    durations = []
    for episode in range(episodes):
        cur_state = env.restart(desiredState)
        mobRob.reset()
        start_time = time.time()
        reason = ''
        episode_rewards = []
        for step in range(steps+1):
            total_num_of_steps += 1
            action = mobRob.act(cur_state)
            actions[episode][step] = action
            # print(action)
            new_state, reward, done = env.step(action, desiredState)
            mobRob.step(cur_state, action, reward, new_state, done)

            cur_state = new_state
            episode_rewards.append(reward)

            if step % learn_every == 0:
                for _ in range(num_learn):
                    mobRob.start_learn()

            if step < steps and done and ~env.collision:
                reason = 'COMPLETED'
                break

            if step == steps: # time budget for episode was overstepped
                reason = 'TIMEOUT  '
                break

            if env.collision:
                reason = 'COLLISION'
                break

        mean_score = np.mean(episode_rewards)
        min_score = np.min(episode_rewards)
        max_score = np.max(episode_rewards)
        total_rewards.append(mean_score)
        duration = time.time() - start_time
        durations.append(duration)
        save_rewards.append([total_rewards[episode], episode])
        eta = np.mean(durations)*(episodes-episode) / 60 / 60
        if eta < 1.0:
            etaString = str(np.round(eta * 60, 2)) + " min"
        else:
            etaString = str(np.round(eta, 2)) + " h"

        print(
            '\rEpisode {}\t{}\tMean episode reward: {:.2f}\tMin: {:.2f}\tMax: {:.2f}\tDuration: {:.2f}\tETA: {}'
                .format(episode, reason, mean_score, min_score, max_score, duration, etaString))

        gc.collect()

    torch.save(mobRob.actor_local.state_dict(), './actor.pth')
    torch.save(mobRob.critic_local.state_dict(), './critic.pth')
    np.save('mean_episode_rewards', save_rewards)

if __name__ == "__main__":
    main()

Typical output

Episode 3179    TIMEOUT Mean episode reward: 0.36   Min: 0.00   Max: 0.69   Duration: 36.86 ETA: 134.16 h

Episode 3180    TIMEOUT Mean episode reward: 0.22   Min: 0.00   Max: 0.72   Duration: 37.39 ETA: 134.12 h

Episode 3181    COLLISION   Mean episode reward: 0.26   Min: -49.50 Max: 0.54   Duration: 29.11 ETA: 134.08 h

Episode 3182    COLLISION   Mean episode reward: -0.39  Min: -50.00 Max: 0.21   Duration: 9.50  ETA: 134.02 h

Episode 3183    COLLISION   Mean episode reward: -0.06  Min: -50.00 Max: 0.32   Duration: 27.10 ETA: 133.98 h

Episode 3184    COLLISION   Mean episode reward: 0.38   Min: -49.32 Max: 0.69   Duration: 37.90 ETA: 133.94 h

Episode 3185    COLLISION   Mean episode reward: -0.52  Min: -50.00 Max: 0.21   Duration: 7.28  ETA: 133.88 h

Position reached!

Speed reached!

Episode 3186    COMPLETED   Mean episode reward: 0.39   Min: 0.00   Max: 80.72  Duration: 37.73 ETA: 133.84 h

Episode 3187    COLLISION   Mean episode reward: 0.36   Min: -49.36 Max: 0.68   Duration: 34.68 ETA: 133.8 h

Episode 3188    COLLISION   Mean episode reward: 0.32   Min: -49.43 Max: 0.62   Duration: 35.35 ETA: 133.76 h

Episode 3189    COLLISION   Mean episode reward: -0.23  Min: -50.00 Max: 0.21   Duration: 16.59 ETA: 133.71 h

Episode 3190    TIMEOUT Mean episode reward: 0.39   Min: 0.00   Max: 0.65   Duration: 38.15 ETA: 133.67 h

Episode 3191    TIMEOUT Mean episode reward: 0.35   Min: 0.00   Max: 0.67   Duration: 37.76 ETA: 133.63 h

Position reached!

Episode 3192    COMPLETED   Mean episode reward: 0.41   Min: 0.00   Max: 30.71  Duration: 36.36 ETA: 133.59 h

Episode 3193    TIMEOUT Mean episode reward: 0.35   Min: 0.00   Max: 0.72   Duration: 36.98 ETA: 133.55 h

Even after a high number of episodes, I cannot see any apparent rise in rewards.


title: PyTorch Extension on Windows - why unresolved Symbols (THFloatTensor etc.)?

body:
PyTorch extensions seem to be experimental.
There is a work around described here:
https://pytorch.org/docs/stable/notes/windows.html#extension

I converted to cpp and added the libs as described:

from torch.utils.ffi import create_extension

ffi = create_extension(
  '_ext.nms',
  headers=headers,
  sources=sources,
  define_macros=defines,
  relative_to=__file__,
  with_cuda=with_cuda,
  extra_objects=extra_objects,
  libraries=["cudart","_C"]
)

It compiles ok, however, I am still getting a lot of unresolved external symbols from the linker:

nms.obj : error LNK2001: unresolved external symbol __imp_THFloatTensor_data
nms.obj : error LNK2001: unresolved external symbol __imp_THByteTensor_fill
nms.obj : error LNK2001: unresolved external symbol __imp_THExpm1
[...]

The _C.lib does not seem to contain the TH... symbols (checked with dumpbin _C.lib /ALL), and I can't find any libTH.lib or similar to link.

Where do I find the right library?
Is there any chance to do this on Windows?


title: Pytorch and numpy broadcasting rules differences?

body:
I've had some experimentation with torch and here is what I've found:

(  5,7,3) and (  5,7,3) -> (  5,7,3) - equal
(5,3,4,1) and (5,3,1,1) -> (5,3,4,1) - one dim is 1
(5,3,4,1) and (  3,4,1) -> (5,3,4,1) - one dim doesn't exist
(5,3,4,1) and (  3,1,1) -> (5,3,4,1) - one dim is 1 and one doesn't exist
(  4,3,2) and (5,4,1,1) -> (5,4,3,2)
(    4,1) and (5,3,1,1) -> (5,3,4,1) - one dim is 1 and two don't exist
(      1) and (5,3,4,2) -> (5,3,4,2) - one dim is 1 and others don't exist
(       ) and (5,3,4,2) -> (5,3,4,2) - scalar and tensor
(      0) and (5,3,2,1) -> (5,3,2,0) - empty tensor and tensor
(    4,1) and (      4) -> (    4,4) - missing dim is 1
(    4,2) and (      4) -> not broadcastable
(  5,2,1) and (5,3,2,1) -> not broadcastable
(5,3,2,1) and (  5,3,2) -> not broadcastable

Pytorch documentation says:

  Many PyTorch operations support NumPy Broadcasting Semantics.

Does that mean that pytorch follows numpy's broadcasting rules and they are the same or should I experiment with numpy also?

The documentation for numpy doesn't list specific rules as the pytourch'es one. And even pytorch'es documentation is not completely accurate, e.g. it says that each tensor should have at least one dimension but torch.empty(3,4) + torch.tensor(42) works fine (scalar has no dimensions).

The rules that I've found are:

Starting at the trailing dimension, dim sizes must either be equal or one of them should be 1 (for missing first dims 1 is assumed).
For each dim the resulting dim is the max of the two.
In-place operations do not allow the in-place tensor to change shape.

It seems like the most reasonable approach to broadcasting and I don't see why it would be different for numpy. But maybe there are some features.

If someone can provide proofs or at least vouch that numpy has the same rules as pytorch then I'd be grateful since won't have to study numpy broadcasting behavior.