title: Vectorize index identification for faster GPU processing

body:
My input is a list of values, data_idx. In the example, the values range from [0, 5].

data_idx = [2, 5, 5, 0, 4, 1, 4, 5, 3, 2, 1, 0, 3, 3, 0]

My desired output, filled_matrix is a tensor of shape max(value) by len(data_idx) where each row, r, of the tensor contains all of indices where data_idx == r and -1 for the rest of the row if the number of matched indices is fewer than len(data_idx)

For example, in the first row, r=0, data_idx==0 at indices [3, 11, 14]. The full output would look like:

filled_matrix = tensor([[ 3, 11, 14, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1],
    [ 5, 10, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1],
    [ 0,  9, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1],
    [ 8, 12, 13, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1],
    [ 4,  6, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1],
    [ 1,  2,  7, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]],
   dtype=torch.int8)

I have working for-loop code that accomplishes my goal.

import torch

max_idx = 6
data_idx = torch.tensor([2, 5, 5, 0, 4, 1, 4, 5, 3, 2, 1, 0, 3, 3, 0]).cuda()

max_number_data_idx = data_idx.shape[0]

filled_matrix = torch.zeros([max_idx, max_number_data_idx], dtype=torch.int8, device='cuda')
filled_matrix.fill_(-1)
for i in range(max_idx):
   same_idx = (data_idx == i).nonzero().flatten()
   filled_matrix[i][:same_idx.shape[0]] = same_idx

Now, I want to speed up this code. Specifically, I want it to be faster on the GPU. In the real scenario, the input, data_idx, can be a list containing millions of values. In that case, e.g 1 M of different values, the GPU will be call 1 M of time which make it very slow. My code is sequential and GPU hate sequential code.

Is there a function which will produce the same result more efficiently? Or a way to vectorize this for-loop?


title: I am having hard time understanding the BCEWITHLOGITLOSS

body: Hello everyone recently i am working on a assignment where i have to predict mask of image using foreground_background image,background image.I used bcewithlogitloss bcz i changed my target value as combination of 1 and 0 where 1 is for foreground and 0 is for background.So my results are pretty good but i am not still convinced why i used bcewithlogitloss.It is a reconstruction problem where i am using probabilty distribution(bcewithlogitloss).SO can anyone plz hepl me why i can use bcewithlogitloss in this example.Thank you


title: PyTorch: why does sort_within_batch impact my LSTM so much?

body:
I'm experienced with Keras and am coming over to PyTorch. I've implemented a super-simple LSTM model for sentiment analysis, but I'm finding that an upstream setting, sort_within_batch in the dataloader, is having a huge impact on my results.

Here is my LSTM sentiment analysis model. It is very simple and avoids any packing/unpacking.

class LSTM(nn.Module):

    def __init__(self, vocab_size, embedding_dim, rnn_dim, output_dim):

        super().__init__()

        self.embedding = nn.Embedding(vocab_size, embedding_dim)
        self.lstm      = nn.LSTM(embedding_dim, rnn_dim)
        self.fc        = nn.Linear(rnn_dim, output_dim)

    def forward(self, text): # Expect text to be [tokens, batch]

        a0          = self.embedding(text)  # a0 is [tokens, batch, emb_dim]
        _, (a1, _)  = self.lstm(a0)         # a1 is [1, batch, rnn_dim]
        a2          = self.fc(a1[0])        # a2 is [batch, 1]

        return a2

Before I start training the model, I have the choice of setting the variable sort_within_batch in the dataloader class to be either True or False.

When I set it to False,

dl_train, dl_val, dl_test = \
    tt.data.BucketIterator.splits((ds_train, ds_val, ds_test),
                                  batch_size=BATCH_SIZE,
                                  sort_within_batch=False,
                                  device=device)

the model does not seem to be leaning anything at all over 5 epochs, as seen by the flat loss and accuracy values:

Epoch 1 took 25 sec. TRAIN [loss:  0.69, acc:  0.50] VAL [loss:  0.69, acc:  0.49]
Epoch 2 took 25 sec. TRAIN [loss:  0.69, acc:  0.51] VAL [loss:  0.69, acc:  0.52]
Epoch 3 took 25 sec. TRAIN [loss:  0.69, acc:  0.50] VAL [loss:  0.70, acc:  0.51]
Epoch 4 took 25 sec. TRAIN [loss:  0.69, acc:  0.51] VAL [loss:  0.72, acc:  0.50]
Epoch 5 took 25 sec. TRAIN [loss:  0.69, acc:  0.50] VAL [loss:  0.73, acc:  0.51]

On the other hand, when I set it to True,

dl_train, dl_val, dl_test = \
    tt.data.BucketIterator.splits((ds_train, ds_val, ds_test),
                                  batch_size=BATCH_SIZE,
                                  sort_within_batch=True,
                                  device=device)

the model exhibits learning during training, and the training time is shorter.

Epoch 1 took 9 sec. TRAIN [loss:  0.69, acc:  0.54] VAL [loss:  0.67, acc:  0.59]
Epoch 2 took 9 sec. TRAIN [loss:  0.64, acc:  0.64] VAL [loss:  0.68, acc:  0.56]
Epoch 3 took 9 sec. TRAIN [loss:  0.54, acc:  0.73] VAL [loss:  0.47, acc:  0.79]
Epoch 4 took 9 sec. TRAIN [loss:  0.35, acc:  0.85] VAL [loss:  0.40, acc:  0.83]
Epoch 5 took 9 sec. TRAIN [loss:  0.26, acc:  0.90] VAL [loss:  0.38, acc:  0.84]

What's going on here? Why is sort_within_batch = True having such a big impact?

Thanks for any help.


title: BERT:Question-Answering - Total number of permissible words/tokens for training

body:
Let's say I want to train BERT with 2 sentences (query-answer) pair against a certain binary label (1,0) for the correctness of the answer,  will BERT let me use 512 words/tokens each for the query and the answer or together(query+answer combined) they should be 512? [510 upon ignoring the [start] and [sep] token]

Thanks in advance!


title: How to select an action from a matrix in Q learning when using multiple frames as input

body:
When using deep q-learning I am trying to capture motion by passing a number of grayscale frames as the input, each with the dimensions 90x90. There will be four 90x90 frames passed in to allow the network to detect motion. The multiple frames should be considered a single state rather than a batch of 4 states, how can I get a vector of actions as a result instead of a matrix?
I am using pytorch and it will return a matrix of 4x7 - a row of actions for each frame. here is the network:
        self.conv1 = Conv2d(self.channels, 32, 8)
        self.conv2 = Conv2d(32, 64, 4)
        self.conv3 = Conv2d(64, 128, 3)
        self.fc1 = Linear(128 * 52 * 52, 64)
        self.fc2 = Linear(64, 32)
        self.output = Linear(32, action_space)


title: nn.Conv2d is faster than nn.Conv3d for the same data size and flops

body:
Given a tensor with the shape of [NxCxDxHxW], one can merge the 2nd and 3rd dims into one dim, after which the tensor will be like [Nx(CxD)xHxW]. The process above is just a reshape changing tensor from 5d to 4d without size reduction. In my case, 3d convolution applied to the [NxCxDxHxW] runs slower than 2d convolution applied to [Nx(CxD)xHxW]. We can easily calculate flops of the two processes above, in which 2d case has more flops than 3d case. My guess is that it's related to the GEMM written in cuDNN. But I cannot understand what makes this happen. Does anyone know what mechanism impacts this?
My speed evaluation code:
cost4x = torch.rand(1,4,5, 312, 96).cuda().float()

cost4x_merge = torch.rand(1,20, 312, 96).cuda().float()

class conv3d(nn.Module):

  def __init__(self, args):

    super(conv3d, self).__init__()

    self.conv = nn.Conv3d(4, 4, 3, 1, 1, bias=False)

  def forward(self, in_t):

    return self.conv(in_t)

class conv3d_merge(nn.Module):

  def __init__(self, args):

    super(conv3d_merge, self).__init__()

    self.conv = nn.Conv2d(20, 20, 3, 1, 1, bias=False)

  def forward(self, in_t):

    return self.conv(in_t)

model3 = conv3d(args).eval().cuda()

model4 = conv3d_merge(args).eval().cuda()

starter, ender = torch.cuda.Event(enable_timing=True), torch.cuda.Event(enable_timing=True)

repetitions = 1000

with torch.no_grad():

  for i in range(repetitions):

    starter.record()

    out = model3(cost4x)

    # out = model4(cost4x_merge)

    ender.record()

    torch.cuda.synchronize()

    curr_time = starter.elapsed_time(ender)

    times.append(curr_time / 1000)

print('Mean time: {:6f}'.format(sum(times[200:]) / (len(times) - 200)))

My evaluation results:
Conv2d: 0.078ms
Conv3d: 0.235ms


title: Passing Tensorboard summaries when saving and loading checkpoints

body:
I am using Tensorboard for visualizing the results, how can I pass the summary when saving a general checkpoint so I can resume visualization from same epoch?
here is the code I am using:
checkpoint = {'epoch': epoch + 1, 'state_dict': model.state_dict(), 'optimizer': optimizer.state_dict()}

def resume_from_checkpoint(fpath='xyz.pth', model, optimizer=None, scheduler=None):

    print('Loading checkpoint from "{}"'.format(fpath))
    checkpoint = load_checkpoint(fpath)
    model.load_state_dict(checkpoint['state_dict'])
    print('Loaded model weights')

    if optimizer is not None and 'optimizer' in checkpoint.keys():
        optimizer.load_state_dict(checkpoint['optimizer'])
        print('Loaded optimizer')
    if scheduler is not None and 'scheduler' in checkpoint.keys():
        scheduler.load_state_dict(checkpoint['scheduler'])
        print('Loaded scheduler')

    epoch = checkpoint['epoch']
    print('Last epoch = {}'.format(epoch))

    if 'rank1' in checkpoint.keys():
        print('Last rank1 = {:.1%}'.format(checkpoint['rank1']))
    return epoch

for epoch in range(num_epochs):
    total_correct = 0
    total_loss = 0
    if epoch % 1 == 0:
        checkpoint = {'epoch': epoch + 1, 'state_dict': model.state_dict(), 'optimizer': optimizer.state_dict()}
        print("saving checkpoint")
        save_checkpoint(checkpoint)
    for i, traindata in enumerate(trainloader):
        images =
        labels =
        optimizer.zero_grad()


title: Using neural networks for portfolio selection

body:
I'm currently trying to build an algorithm to maximize terminal wealth of a portfolio. I am using the REINFORCE with baseline algorithm present in Sutton and Barto (2018). I have one neural network for the policy, which has current wealth and time left on investment horizon as inputs, and outputs two values: a mean and standard deviation of a normal distribution. The discounted dollar amount invested in the risky asset is then sampled from this distribution. I have another network for the value function (same inputs but outputs state value). I have solved the problem analytically and my value network converges to the optimal solution well. My policy network does not which leads me to believe that I could improve the architecture of the network to 'help' it find the optimal solution. I am reasonably new to pytorch and neural networks and so would appreciate ideas as to how i could do this. My policy network is below, it has two hidden layers with 32 nodes each. I have also played around with the learning rates and it does not seem to help too much. Thanks!
class PolicyNetwork(nn.Module):
    ''' Neural Network for the policy, which is taken to be normally distributed hence
    this network returns a mean and variance '''
    def __init__(self, lr, input_dims, fc1_dims, fc2_dims, n_returns):
        super(PolicyNetwork, self).__init__()
        self.input_dims = input_dims
        self.fc1_dims = fc1_dims
        self.fc2_dims = fc2_dims
        self.n_returns = n_returns
        self.lr = lr
        self.fc1 = nn.Linear(*self.input_dims, self.fc1_dims) # inputs should be wealth and time to maturity
        self.fc2 = nn.Linear(self.fc1_dims,self.fc2_dims)
        self.fc3 = nn.Linear(self.fc2_dims,n_returns) # returns mean and sd of normal dist
        self.optimizer = optim.Adam(self.parameters(), lr = lr)

    def forward(self, observation):
        state = torch.Tensor(observation).float().unsqueeze(0)
        x = F.relu(self.fc1(state))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        first_slice = x[:,0]
        second_slice = x[:,1]
        tuple_of_activated_parts = (
                first_slice, # let mean be negative
                #F.relu(first_slice), # make sure mean is positive
                #torch.sigmoid(second_slice) # make sure sd is positive
                F.softplus(second_slice) # make sd positive but dont trap below 1
                )
        out = torch.cat(tuple_of_activated_parts, dim=-1)
        return out


title: Multiple input linear regression cannot predict outputs of large variance

body:
I succeeded to build a linear regression neural network with 1 inputs and 1 outputs.
I am building a linear regression neural network with 5 inputs and 1 outputs now.
Here is the formula:
y = 3e + d^2 + 9c + 11b^6 + a + 19
However, no matter how many neurons, epochs and hidden layers I use, I cannot predict the a good result.
The predicted outputs are always within a small range. However, there are large variance among the expected outputs.
Predicted output vs Expected output
I guess it may be because of the choice of activation function, loss function and optimizer.
If not, multiple input neural network may need alternative method to build.
Here is my code:
import torch
import torch.nn as nn    #neural network model
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import torch.nn.functional as F
from torch.autograd import Variable
from sklearn.preprocessing import MinMaxScaler
from pickle import dump

#Load datasets
dataset = pd.read_csv('testB_200.csv')

X = dataset.iloc[:, :-1].values
Y = dataset.iloc[:, -1:].values
X_scaler = MinMaxScaler()
Y_scaler = MinMaxScaler()
print(X_scaler.fit(X))
print(Y_scaler.fit(Y))
X = X_scaler.transform(X)
Y = Y_scaler.transform(Y)

#save the scaler
dump(X_scaler, open('X_scaler.pkl', 'wb'))
dump(Y_scaler, open('Y_scaler.pkl', 'wb'))

train = int((len(dataset)+1)*0.8)
test = train + 1
print(train)
print(test)
x_temp_train = X[:train]
y_temp_train = Y[:train]
x_temp_test = X[test:]
y_temp_test = Y[test:]

X_train = torch.FloatTensor(x_temp_train)
Y_train = torch.FloatTensor(y_temp_train)
X_test = torch.FloatTensor(x_temp_test)
Y_test = torch.FloatTensor(y_temp_test)

D_in = 5 # D_in is input features
H = 12 # H is hidden dimension
H2 =8
H3 =4

D_out = 1 # D_out is output features.

#Define a Artifical Neural Network model
class Net(nn.Module):
#------------------3 hidden Layers------------------------------
    def __init__(self, D_in, H, H2, H3, D_out):
        super(Net, self).__init__()

        self.linear1 = nn.Linear(D_in, H)
        self.linear2 = nn.Linear(H, H2)
        self.linear3 = nn.Linear(H2, H3)
        self.linear4 = nn.Linear(H3, D_out)

    def forward(self, x):
        #activation function should be used here  e.g: hidden = F.relu(...)
        h_relu = self.linear1(x).clamp(min=0) #min=0 is like ReLU
        middle = self.linear2(h_relu).clamp(min=0)
        middle2 = self.linear3(middle).clamp(min=0)
        prediction = self.linear4(middle2)

        return prediction
model = Net(D_in, H, H2, H3, D_out)
print(model)

#Define a Loss function and optimizer
criterion = torch.nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.2) #2e-7, lr=learning rate=0.2

#Training model
inputs = Variable(X_train)
outputs = Variable(Y_train)
inputs_val = Variable(X_test)
outputs_val = Variable(Y_test)
loss_values = []
val_values = []
epoch = []
epoch_value=25

for i in range(epoch_value):
    for phase in ['train', 'val']:
        if phase == 'train':
            #print('train loss')
            model.train()  # Set model to training mode
            prediction = model(inputs)
            loss = criterion(prediction, outputs)
            #print(loss)
            loss_values.append(loss.item())
            optimizer.zero_grad() #zero the parameter gradients
            epoch.append(i)
            loss.backward()       #compute gradients(dloss/dx)
            optimizer.step()      #updates the parameters
        elif phase == 'val':
            #print('validation loss')
            model.eval()   # Set model to evaluate mode
            prediction_val = model(inputs_val)
            loss_val = criterion(prediction_val, outputs_val)
            #print(loss_val)
            val_values.append(loss_val.item())
            optimizer.zero_grad() #zero the parameter gradients

torch.save(model.state_dict(), 'formula2.pth')  #save model

#Plot train_loss vs validation loss
plt.plot(epoch,loss_values)
plt.plot(epoch, val_values)
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train','validation'], loc='upper left')
plt.show()

#plot prediciton vs expected value
prediction_val = prediction_val.detach().numpy()
prediction_val = Y_scaler.inverse_transform(prediction_val)
#print('predict')
#print(prediction_val)
Y_test = Y_scaler.inverse_transform(Y_test)
#print('test')
#print(Y_test)
plt.plot(Y_test)
plt.plot(prediction_val)
plt.legend(['expected','predict'], loc='upper left')
plt.show()

Model Loss vs Validation Loss
Validation vs Expected outputs
Thanks for your time.


title: How to get model parameters from traced model of TorchScript?

body:
As far as I know, torch.jit.trace can be used to trace torch module, such as t_model = torch.jit.script(model, torch.ones(100, dtype=torch.float))
When I get the serialized model t_model. How can I get the model parameters?


title: Convert tensor list into numpy then sum the numbers

body:
I would like to convert the tensor list into a numpy array/list then sum it and multiply by the number of epochs. I have tried to use MSE_for_all_epochs = sum(np.MSE_for_all_epochs) but getting the following error message: Traceback (most recent call last):
File "train3_Eval.py", line 177, in
MSE_for_all_epochs = sum(np.MSE_for_all_epochs)
AttributeError: module 'numpy' has no attribute 'MSE_for_all_epochs'
        mean_test_error = mean_test_error.detach() / len(loader_test)
        #store_MAE += mean_test_error
        MAE_for_all_epochs.append(mean_test_error)

        mse = math.sqrt(loss / len(loader_test))
        #store_MSE +=mse
        MSE_for_all_epochs.append(mse)

        print("Test count error: %f" % mean_test_error)
        print("MSE: %f" % mse)

MSE_for_all_epochs = sum(np.MSE_for_all_epochs)
MAE_for_all_epochs = sum(np.MAE_for_all_epochs)
print(MAE_for_all_epochs)
print(MSE_for_all_epochs)

model_mae= MSE_for_all_epochs / epoch
model_mse= MAE_for_all_epochs / epoch
print("Model MAE: %f" % model_mae)
print("Model MSE: %f" % model_mse)

Model output this:
tensor(1445.2251)
1602.6452224985646
Model MAE: 534.215074
Model MSE: 481.741699


title: Pytorch: loss is not changing

body:
I created a neural network in PyTorch. My loss function is a weighted negative log-likelihood. The weights are determined by the output of my neural network and must be fixed. It means the weights depend on the output of the neural network but must be fixed so the network only calculates the gradient of log part and not the weights. Here is my code:
import torch
import torch.nn as nn

def extended_cumsum(x):
    return torch.cat([torch.zeros(x.size()[1:3]).unsqueeze(0), torch.cumsum(x, 0)], 0)

def is_positive(x):
    x_sign = torch.sign(x)
    return torch.where(x_sign < 0, torch.zeros_like(x_sign), x_sign)

def coupling_transform(x1, x2, nn_output, k):
    nn_output_normalized = torch.softmax(nn_output, 0)
    bins_weights = torch.ones_like(nn_output_normalized)/k
    knots_xs = extended_cumsum(bins_weights)
    prev_bins = is_positive(x2.unsqueeze(0) - knots_xs)
    current_bins = prev_bins[:-1,:,:] - prev_bins[1:,:,:]
    q_sum = torch.sum(prev_bins[1:,:,:]*nn_output_normalized, 0)
    q_current = torch.sum(current_bins*nn_output_normalized, 0)
    w_sum = torch.sum(prev_bins[1:,:,:]*bins_weights, 0)
    c_values = q_sum + k*(x2 - w_sum)*q_current
    log_det = torch.log(torch.prod(k*q_current, 1))
    return x1, c_values, log_det

def flipping_dims(n, d1, d2):
    dims = []
    for i in range(n):
        if i%2 == 0:
            dims.append(d1)
        else:
            dims.append(d2)

    return dims

class Linear(nn.Module):
    def __init__(self, d1, d2, k, hidden):
        super().__init__()
        self.d1, self.d2, self.k = d1, d2, k
        self.net = nn.Sequential(nn.Linear(d1, hidden),
                            nn.ReLU(),
                            nn.Linear(hidden, hidden),
                            nn.ReLU(),
                            nn.Linear(hidden, hidden),
                            nn.ReLU(),
                            nn.Linear(hidden, k*d2))

    def forward(self, x, log_prob, flip=False):
        x1, x2 = x[:, :self.d1], x[:, self.d1:]

        if flip:
            x1, x2 = x2, x1

        z1, z2, log_det = coupling_transform(x1, x2, torch.reshape(self.net(x1), (self.k, -1, self.d2)), self.k)
        if flip:
            z1, z2 = z2, z1

        z = torch.cat([z1, z2], 1)
        return z, log_prob - log_det

class stacked_Linear(nn.Module):
    def __init__(self, d1, d2, k, hidden, n):
        super().__init__()
        self.layers = nn.ModuleList([
            Linear(_d1, _d2, k, hidden=hidden) for _, _d1, _d2 in zip(range(n), flipping_dims(n, d1, d2), flipping_dims(n, d1, d2)[::-1])
        ])

        self.flips = [True if i%2 else False for i in range(n)]

    def forward(self, x, log_prev_prob):

        for layer, f in zip(self.layers, self.flips):
            x, log_prob = layer(x, log_prev_prob, flip=f)
            log_prev_prob = log_prob

        return x, log_prob

def f(x):
    return torch.prod(torch.exp(x), 1)

def my_loss(weights, log_prob):
    loss = -torch.mean(weights*log_prob)
    return loss

model = stacked_Linear(3, 3, 32, 16, 4)
optim = torch.optim.Adam(model.parameters(), lr=1e-4)

losses = []
for _ in range(100):
    x = torch.rand(1000, 6)
    optim.zero_grad()
    z, log_prob = model(x, torch.zeros(1000))
    f_values = f(z)
    weights = f_values/torch.exp(log_prob)
    loss = my_loss(weights, log_prob)
    losses.append(loss)

    loss.backward()

But the loss value doesn't decrease and it doesn't change if I fix x:
losses = []
x = torch.rand(1000, 6)
for _ in range(100):
    optim.zero_grad()
    z, log_prob = model(x, torch.zeros(1000))
    f_values = f(z)
    weights = f_values/torch.exp(log_prob)
    loss = my_loss(weights, log_prob)
    losses.append(loss)

    loss.backward()

[tensor(-0.1160, grad_fn=<NegBackward>),
 tensor(-0.1160, grad_fn=<NegBackward>),
 tensor(-0.1160, grad_fn=<NegBackward>),
 tensor(-0.1160, grad_fn=<NegBackward>),
 tensor(-0.1160, grad_fn=<NegBackward>), ...]


I saw similar topic but I don't get why my code contains an inplace operation when I have the boolean self.noisy set to True. More explanation after the code
code:
l = self.backward()

with:
def backward(self):
     transitions = self.replay_memory.sample(self.batch_size)

        batch = Transition(*zip(*transitions))

        a_batch = torch.cat(batch.action).to(self.device) # [BS x 1]
        cs_batch = torch.stack(batch.state).to(self.device)    # [BS x state_size]
        ns_batch = torch.stack(batch.next_state).to(self.device) # [BS x state_size]
        r_batch = torch.tensor(np.expand_dims(np.array(batch.reward),1), dtype=torch.float).to(self.device)  # [BS x 1]

# STEP 2: PREDICTIONS
        pred = self.calc_pred(cs_batch, a_batch)

        # STEP 3: TARGETS
        with torch.no_grad():
            target = self.calc_target(ns_batch, r_batch)

        # STEP 4: LOSS
        loss = self.calc_loss(pred, target)   # [BS x 1]

        loss = loss.mean()

    # STEP 5: TRAIN

       self.optimizer.zero_grad()
       loss.backward()

        self.optimizer.step()
 return loss

With

    def calc_pred(self, cs_batch, a_batch):

        if self.noisy:

                self.network.sample_noise()

        pred = self.network(cs_batch).gather(1, a_batch)

        return pred

And also:
 def calc_target(self, ns_batch, r_batch):

    if self.double:

                if self.noisy:
                    self.network.sample_noise()
                    self.target_network.sample_noise()
                next_Q = self.network(ns_batch)
                max_a = next_Q.max(1)[1].unsqueeze(1)
                Q_target = self.target_network(ns_batch).gather(1, max_a)
    target = r_batch + (self.gamma ** self.multi_step_n) * Q_target

return target

and loss function:

def calc_loss(self, pred, target):
     loss = F.mse_loss(pred, target, reduction='none')

 return loss

This works if am not using a noisy network. However, when using noisy (self.noisy = True)  it returns the following error:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [5]] is at version 1003; expected version 1002 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

However, I do not change any variables in my code whenever I use my noisy network? So I do not get why this gives an inplace error. I have used the noisy network also in another piece of code were I do not get the error, so I do not think the error is in that part of the code. For reference I post it anyway:
def initialize_weights_he(m):
    if isinstance(m, nn.Linear):
        torch.nn.init.kaiming_uniform_(m.weight)
        if m.bias is not None:
            torch.nn.init.constant_(m.bias, 0)

class NoisyLinear(nn.Module):
  def __init__(self, in_features, out_features, std_init=0.5):
    super(NoisyLinear, self).__init__()
    self.in_features = in_features
    self.out_features = out_features
    self.std_init = std_init

    self.weight_mu = nn.Parameter(torch.empty(out_features, in_features))
    self.weight_sigma = nn.Parameter(torch.empty(out_features, in_features))
    self.register_buffer('weight_epsilon', torch.empty(out_features, in_features))

    self.bias_mu = nn.Parameter(torch.empty(out_features))
    self.bias_sigma = nn.Parameter(torch.empty(out_features))
    self.register_buffer('bias_epsilon', torch.empty(out_features))

    self.reset_parameters()
    self.sample_noise()

  def reset_parameters(self):
    mu_range = 1 / math.sqrt(self.in_features)
    self.weight_mu.data.uniform_(-mu_range, mu_range)
    self.weight_sigma.data.fill_(self.std_init / math.sqrt(self.in_features))
    self.bias_mu.data.uniform_(-mu_range, mu_range)
    self.bias_sigma.data.fill_(self.std_init / math.sqrt(self.out_features))

  def _scale_noise(self, size):
    x = torch.randn(size)
    return x.sign().mul_(x.abs().sqrt_())

  def sample_noise(self):
    epsilon_in = self._scale_noise(self.in_features)
    epsilon_out = self._scale_noise(self.out_features)
    self.weight_epsilon.copy_(epsilon_out.ger(epsilon_in))
    self.bias_epsilon.copy_(epsilon_out)

  def forward(self, x):
    if self.training:
      return F.linear(x,
                      self.weight_mu + self.weight_sigma * self.weight_epsilon,
                      self.bias_mu + self.bias_sigma * self.bias_epsilon)
    else:
      return F.linear(x,
                      self.weight_mu,
                      self.bias_mu)

class DQN(nn.Module):
    def __init__(self, num_inputs, fc1_dims, fc2_dims, num_actions, use_duel=False, use_noisy_net=False):
        super(DQN, self).__init__()
        self.use_duel = use_duel
        self.use_noisy_net = use_noisy_net

        if use_noisy_net:
            Linear = NoisyLinear
        else:
            Linear = nn.Linear

        self.fc1 = Linear(num_inputs, fc1_dims)
        if self.use_duel:
            self.fcA = Linear(fc1_dims, fc2_dims)
            self.A = Linear(fc2_dims, num_actions)

            self.fcV = Linear(fc1_dims, fc2_dims)
            self.V = Linear(fc2_dims, 1)
        else:
            self.fcQ = Linear(fc1_dims, fc2_dims)
            self.Q = Linear(fc2_dims, num_actions)

        self._init_params()
        self.train()

    def forward(self, x):
        x = F.relu(self.fc1(x))
        if self.use_duel:
            A = F.relu(self.fcA(x))
            A = self.A(A)
            V = F.relu(self.fcV(x))
            V = self.V(V)
            return V + A - A.mean()
        else:
            Q = F.relu(self.fcQ(x))
            return self.Q(Q)

    def sample_noise(self):
        if self.use_noisy_net:
            for m in self.modules():
                if isinstance(m, NoisyLinear):
                    m.sample_noise()

    def _init_params(self):
        for m in self.modules():
            m.apply(lambda x: initialize_weights_he(x))


title: Pytorch not utilizing GPU despite all settings seemingly correct

body:
I've got a CUDA 10.2 on a 2070 SUPER card and
import torch

torch.cuda.is_available()

returns true.
torch.cuda.get_device_name(0)

returns 2070 SUPER correctly.
The code I'm running specifies using CUDA:
import shutil
import os
import time
from datetime import datetime
import argparse
import numpy as np
from tqdm import tqdm

import torch
import torch.nn as nn
import torch.optim as optim
from torch.autograd import Variable
from torchsample.transforms import RandomRotate, RandomTranslate, RandomFlip, ToTensor, Compose, RandomAffine
from torchvision import transforms
import torch.nn.functional as F
from tensorboardX import SummaryWriter

from dataloader import MRDataset
import model

from sklearn import metrics

def train_model(model, train_loader, epoch, num_epochs, optimizer, writer, current_lr, log_every=100):
    _ = model.train()

    if torch.cuda.is_available():
        model.cuda()

    y_preds = []
    y_trues = []
    losses = []

    for i, (image, label, weight) in enumerate(train_loader):
        optimizer.zero_grad()

        if torch.cuda.is_available():
            image = image.cuda()
            label = label.cuda()
            weight = weight.cuda()

        label = label[0]
        weight = weight[0]

        prediction = model.forward(image.float())

        loss = torch.nn.BCEWithLogitsLoss(weight=weight)(prediction, label)
        loss.backward()
        optimizer.step()

        loss_value = loss.item()
        losses.append(loss_value)

        probas = torch.sigmoid(prediction)

        y_trues.append(int(label[0][1]))
        y_preds.append(probas[0][1].item())

        try:
            auc = metrics.roc_auc_score(y_trues, y_preds)
        except:
            auc = 0.5

        writer.add_scalar('Train/Loss', loss_value,
                          epoch * len(train_loader) + i)
        writer.add_scalar('Train/AUC', auc, epoch * len(train_loader) + i)

        if (i % log_every == 0) & (i > 0):
            print('''[Epoch: {0} / {1} |Single batch number : {2} / {3} ]| avg train loss {4} | train auc : {5} | lr : {6}'''.
                  format(
                      epoch + 1,
                      num_epochs,
                      i,
                      len(train_loader),
                      np.round(np.mean(losses), 4),
                      np.round(auc, 4),
                      current_lr
                  )
                  )

    writer.add_scalar('Train/AUC_epoch', auc, epoch + i)

    train_loss_epoch = np.round(np.mean(losses), 4)
    train_auc_epoch = np.round(auc, 4)
    return train_loss_epoch, train_auc_epoch

def evaluate_model(model, val_loader, epoch, num_epochs, writer, current_lr, log_every=20):
    _ = model.eval()

    if torch.cuda.is_available():
        model.cuda()

    y_trues = []
    y_preds = []
    losses = []

    for i, (image, label, weight) in enumerate(val_loader):

        if torch.cuda.is_available():
            image = image.cuda()
            label = label.cuda()
            weight = weight.cuda()

        label = label[0]
        weight = weight[0]

        prediction = model.forward(image.float())

        loss = torch.nn.BCEWithLogitsLoss(weight=weight)(prediction, label)

        loss_value = loss.item()
        losses.append(loss_value)

        probas = torch.sigmoid(prediction)

        y_trues.append(int(label[0][1]))
        y_preds.append(probas[0][1].item())

        try:
            auc = metrics.roc_auc_score(y_trues, y_preds)
        except:
            auc = 0.5

        writer.add_scalar('Val/Loss', loss_value, epoch * len(val_loader) + i)
        writer.add_scalar('Val/AUC', auc, epoch * len(val_loader) + i)

        if (i % log_every == 0) & (i > 0):
            print('''[Epoch: {0} / {1} |Single batch number : {2} / {3} ] | avg val loss {4} | val auc : {5} | lr : {6}'''.
                  format(
                      epoch + 1,
                      num_epochs,
                      i,
                      len(val_loader),
                      np.round(np.mean(losses), 4),
                      np.round(auc, 4),
                      current_lr
                  )
                  )

    writer.add_scalar('Val/AUC_epoch', auc, epoch + i)

    val_loss_epoch = np.round(np.mean(losses), 4)
    val_auc_epoch = np.round(auc, 4)
    return val_loss_epoch, val_auc_epoch

def get_lr(optimizer):
    for param_group in optimizer.param_groups:
        return param_group['lr']

def run(args):
    log_root_folder = "./logs/{0}/{1}/".format(args.task, args.plane)
    if args.flush_history == 1:
        objects = os.listdir(log_root_folder)
        for f in objects:
            if os.path.isdir(log_root_folder + f):
                shutil.rmtree(log_root_folder + f)

    now = datetime.now()
    logdir = log_root_folder + now.strftime("%Y%m%d-%H%M%S") + "/"
    os.makedirs(logdir)

    writer = SummaryWriter(logdir)

    augmentor = Compose([
        transforms.Lambda(lambda x: torch.Tensor(x)),
        RandomRotate(25),
        RandomTranslate([0.11, 0.11]),
        RandomFlip(),
        transforms.Lambda(lambda x: x.repeat(3, 1, 1, 1).permute(1, 0, 2, 3)),
    ])

    train_dataset = MRDataset('./data/', args.task,
                              args.plane, transform=augmentor, train=True)
    train_loader = torch.utils.data.DataLoader(
        train_dataset, batch_size=1, shuffle=True, num_workers=0, drop_last=False)

    validation_dataset = MRDataset(
        './data/', args.task, args.plane, train=False)
    validation_loader = torch.utils.data.DataLoader(
        validation_dataset, batch_size=1, shuffle=-True, num_workers=0, drop_last=False)

    mrnet = model.MRNet()

    if torch.cuda.is_available():
        mrnet = mrnet.cuda()

    optimizer = optim.Adam(mrnet.parameters(), lr=args.lr, weight_decay=0.1)

    if args.lr_scheduler == "plateau":
        scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(
            optimizer, patience=3, factor=.3, threshold=1e-4, verbose=True)
    elif args.lr_scheduler == "step":
        scheduler = torch.optim.lr_scheduler.StepLR(
            optimizer, step_size=3, gamma=args.gamma)

    best_val_loss = float('inf')
    best_val_auc = float(0)

    num_epochs = args.epochs
    iteration_change_loss = 0
    patience = args.patience
    log_every = args.log_every

    t_start_training = time.time()

    for epoch in range(num_epochs):
        current_lr = get_lr(optimizer)

        t_start = time.time()

        train_loss, train_auc = train_model(
            mrnet, train_loader, epoch, num_epochs, optimizer, writer, current_lr, log_every)
        val_loss, val_auc = evaluate_model(
            mrnet, validation_loader, epoch, num_epochs, writer, current_lr)

        if args.lr_scheduler == 'plateau':
            scheduler.step(val_loss)
        elif args.lr_scheduler == 'step':
            scheduler.step()

        t_end = time.time()
        delta = t_end - t_start

        print("train loss : {0} | train auc {1} | val loss {2} | val auc {3} | elapsed time {4} s".format(
            train_loss, train_auc, val_loss, val_auc, delta))

        iteration_change_loss += 1
        print('-' * 30)

        if val_auc > best_val_auc:
            best_val_auc = val_auc
            if bool(args.save_model):
                file_name = f'model_{args.prefix_name}_{args.task}_{args.plane}_val_auc_{val_auc:0.4f}_train_auc_{train_auc:0.4f}_epoch_{epoch+1}.pth'
                for f in os.listdir('./models/'):
                    if (args.task in f) and (args.plane in f) and (args.prefix_name in f):
                        os.remove(f'./models/{f}')
                torch.save(mrnet, f'./models/{file_name}')

        if val_loss < best_val_loss:
            best_val_loss = val_loss
            iteration_change_loss = 0

        if iteration_change_loss == patience:
            print('Early stopping after {0} iterations without the decrease of the val loss'.
                  format(iteration_change_loss))
            break

    t_end_training = time.time()
    print(f'training took {t_end_training - t_start_training} s')

def parse_arguments():
    parser = argparse.ArgumentParser()
    parser.add_argument('-t', '--task', type=str, required=True,
                        choices=['abnormal', 'acl', 'meniscus'])
    parser.add_argument('-p', '--plane', type=str, required=True,
                        choices=['sagittal', 'coronal', 'axial'])
    parser.add_argument('--prefix_name', type=str, required=True)
    parser.add_argument('--augment', type=int, choices=[0, 1], default=1)
    parser.add_argument('--lr_scheduler', type=str,
                        default='plateau', choices=['plateau', 'step'])
    parser.add_argument('--gamma', type=float, default=0.5)
    parser.add_argument('--epochs', type=int, default=50)
    parser.add_argument('--lr', type=float, default=1e-5)
    parser.add_argument('--flush_history', type=int, choices=[0, 1], default=0)
    parser.add_argument('--save_model', type=int, choices=[0, 1], default=1)
    parser.add_argument('--patience', type=int, default=5)
    parser.add_argument('--log_every', type=int, default=100)
    args = parser.parse_args()
    return args

if __name__ == "__main__":
    args = parse_arguments()
    run(args)

However, running nvidia-smi --query-gpu=index,gpu_name,memory.total,memory.used,memory.free,temperature.gpu,pstate,utilization.gpu,utilization.memory --format=csv
returns 0, GeForce RTX 2070 SUPER, 8192 MiB, 2774 MiB, 5418 MiB, 53, P2, 9 %, 5 % while the model is training. This is not utilizing the GPU at all.
Anyone has any idea what is wrong here?
EDIT: As per the comment I've received I have tried to more explicitly tell the model to only use CUDA. However to no avail. Here's the new code:
import shutil
import os
import time
from datetime import datetime
import argparse
import numpy as np
from tqdm import tqdm

import torch
import torch.nn as nn
import torch.optim as optim
from torch.autograd import Variable
from torchsample.transforms import RandomRotate, RandomTranslate, RandomFlip, ToTensor, Compose, RandomAffine
from torchvision import transforms
import torch.nn.functional as F
from tensorboardX import SummaryWriter

from dataloader import MRDataset
import model

from sklearn import metrics

torch.cuda.is_available()

def train_model(model, train_loader, epoch, num_epochs, optimizer, writer, current_lr, log_every=100):
    _ = model.train()

    device = torch.device('cuda:0')

    y_preds = []
    y_trues = []
    losses = []

    for i, (image, label, weight) in enumerate(train_loader):
        optimizer.zero_grad()

        image = image.to(device)
        label = label.to(device)
        weight = weight.to(device)

        label = label[0]
        weight = weight[0]

        prediction = model.forward(image.float())

        loss = torch.nn.BCEWithLogitsLoss(weight=weight)(prediction, label)
        loss.backward()
        optimizer.step()

        loss_value = loss.item()
        losses.append(loss_value)

        probas = torch.sigmoid(prediction)

        y_trues.append(int(label[0][1]))
        y_preds.append(probas[0][1].item())

        try:
            auc = metrics.roc_auc_score(y_trues, y_preds)
        except:
            auc = 0.5

        writer.add_scalar('Train/Loss', loss_value,
                          epoch * len(train_loader) + i)
        writer.add_scalar('Train/AUC', auc, epoch * len(train_loader) + i)

        if (i % log_every == 0) & (i > 0):
            print('''[Epoch: {0} / {1} |Single batch number : {2} / {3} ]| avg train loss {4} | train auc : {5} | lr : {6}'''.
                  format(
                      epoch + 1,
                      num_epochs,
                      i,
                      len(train_loader),
                      np.round(np.mean(losses), 4),
                      np.round(auc, 4),
                      current_lr
                  )
                  )

    writer.add_scalar('Train/AUC_epoch', auc, epoch + i)

    train_loss_epoch = np.round(np.mean(losses), 4)
    train_auc_epoch = np.round(auc, 4)
    return train_loss_epoch, train_auc_epoch

def evaluate_model(model, val_loader, epoch, num_epochs, writer, current_lr, log_every=20):
    _ = model.eval()

    model.cuda()

    y_trues = []
    y_preds = []
    losses = []

    for i, (image, label, weight) in enumerate(val_loader):

        image = image.to(device)
        label = label.to(device)
        weight = weight.to(device)

        label = label[0]
        weight = weight[0]

        prediction = model.forward(image.float())

        loss = torch.nn.BCEWithLogitsLoss(weight=weight)(prediction, label)

        loss_value = loss.item()
        losses.append(loss_value)

        probas = torch.sigmoid(prediction)

        y_trues.append(int(label[0][1]))
        y_preds.append(probas[0][1].item())

        try:
            auc = metrics.roc_auc_score(y_trues, y_preds)
        except:
            auc = 0.5

        writer.add_scalar('Val/Loss', loss_value, epoch * len(val_loader) + i)
        writer.add_scalar('Val/AUC', auc, epoch * len(val_loader) + i)

        if (i % log_every == 0) & (i > 0):
            print('''[Epoch: {0} / {1} |Single batch number : {2} / {3} ] | avg val loss {4} | val auc : {5} | lr : {6}'''.
                  format(
                      epoch + 1,
                      num_epochs,
                      i,
                      len(val_loader),
                      np.round(np.mean(losses), 4),
                      np.round(auc, 4),
                      current_lr
                  )
                  )

    writer.add_scalar('Val/AUC_epoch', auc, epoch + i)

    val_loss_epoch = np.round(np.mean(losses), 4)
    val_auc_epoch = np.round(auc, 4)
    return val_loss_epoch, val_auc_epoch

def get_lr(optimizer):
    for param_group in optimizer.param_groups:
        return param_group['lr']

def run(args):
    log_root_folder = "./logs/{0}/{1}/".format(args.task, args.plane)
    if args.flush_history == 1:
        objects = os.listdir(log_root_folder)
        for f in objects:
            if os.path.isdir(log_root_folder + f):
                shutil.rmtree(log_root_folder + f)

    now = datetime.now()
    logdir = log_root_folder + now.strftime("%Y%m%d-%H%M%S") + "/"
    os.makedirs(logdir)

    writer = SummaryWriter(logdir)

    augmentor = Compose([
        transforms.Lambda(lambda x: torch.Tensor(x)),
        RandomRotate(25),
        RandomTranslate([0.11, 0.11]),
        RandomFlip(),
        transforms.Lambda(lambda x: x.repeat(3, 1, 1, 1).permute(1, 0, 2, 3)),
    ])

    train_dataset = MRDataset('./data/', args.task,
                              args.plane, transform=augmentor, train=True)
    train_loader = torch.utils.data.DataLoader(
        train_dataset, batch_size=1, shuffle=True, num_workers=0, drop_last=False)

    validation_dataset = MRDataset(
        './data/', args.task, args.plane, train=False)
    validation_loader = torch.utils.data.DataLoader(
        validation_dataset, batch_size=1, shuffle=-True, num_workers=0, drop_last=False)

    mrnet = model.MRNet()

    mrnet = mrnet.cuda()

    optimizer = optim.Adam(mrnet.parameters(), lr=args.lr, weight_decay=0.1)

    if args.lr_scheduler == "plateau":
        scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(
            optimizer, patience=3, factor=.3, threshold=1e-4, verbose=True)
    elif args.lr_scheduler == "step":
        scheduler = torch.optim.lr_scheduler.StepLR(
            optimizer, step_size=3, gamma=args.gamma)

    best_val_loss = float('inf')
    best_val_auc = float(0)

    num_epochs = args.epochs
    iteration_change_loss = 0
    patience = args.patience
    log_every = args.log_every

    t_start_training = time.time()

    for epoch in range(num_epochs):
        current_lr = get_lr(optimizer)

        t_start = time.time()

        train_loss, train_auc = train_model(
            mrnet, train_loader, epoch, num_epochs, optimizer, writer, current_lr, log_every)
        val_loss, val_auc = evaluate_model(
            mrnet, validation_loader, epoch, num_epochs, writer, current_lr)

        if args.lr_scheduler == 'plateau':
            scheduler.step(val_loss)
        elif args.lr_scheduler == 'step':
            scheduler.step()

        t_end = time.time()
        delta = t_end - t_start

        print("train loss : {0} | train auc {1} | val loss {2} | val auc {3} | elapsed time {4} s".format(
            train_loss, train_auc, val_loss, val_auc, delta))

        iteration_change_loss += 1
        print('-' * 30)

        if val_auc > best_val_auc:
            best_val_auc = val_auc
            if bool(args.save_model):
                file_name = f'model_{args.prefix_name}_{args.task}_{args.plane}_val_auc_{val_auc:0.4f}_train_auc_{train_auc:0.4f}_epoch_{epoch+1}.pth'
                for f in os.listdir('./models/'):
                    if (args.task in f) and (args.plane in f) and (args.prefix_name in f):
                        os.remove(f'./models/{f}')
                torch.save(mrnet, f'./models/{file_name}')

        if val_loss < best_val_loss:
            best_val_loss = val_loss
            iteration_change_loss = 0

        if iteration_change_loss == patience:
            print('Early stopping after {0} iterations without the decrease of the val loss'.
                  format(iteration_change_loss))
            break

    t_end_training = time.time()
    print(f'training took {t_end_training - t_start_training} s')

def parse_arguments():
    parser = argparse.ArgumentParser()
    parser.add_argument('-t', '--task', type=str, required=True,
                        choices=['abnormal', 'acl', 'meniscus'])
    parser.add_argument('-p', '--plane', type=str, required=True,
                        choices=['sagittal', 'coronal', 'axial'])
    parser.add_argument('--prefix_name', type=str, required=True)
    parser.add_argument('--augment', type=int, choices=[0, 1], default=1)
    parser.add_argument('--lr_scheduler', type=str,
                        default='plateau', choices=['plateau', 'step'])
    parser.add_argument('--gamma', type=float, default=0.5)
    parser.add_argument('--epochs', type=int, default=50)
    parser.add_argument('--lr', type=float, default=1e-5)
    parser.add_argument('--flush_history', type=int, choices=[0, 1], default=0)
    parser.add_argument('--save_model', type=int, choices=[0, 1], default=1)
    parser.add_argument('--patience', type=int, default=5)
    parser.add_argument('--log_every', type=int, default=100)
    args = parser.parse_args()
    return args

if __name__ == "__main__":
    args = parse_arguments()
    run(args)


title: how to convert outlogits to tokens?

body:
i have a forward function in allenNlp given by :
  def forward(self, input_tokens, output_tokens):
    '''
    This is the main process of the Model where the actual computation happens.
    Each Instance is fed to the forward method.
    It takes dicts of tensors as input, with same keys as the fields in your Instance (input_tokens, output_tokens)
    It outputs the results of predicted tokens and the evaluation metrics as a dictionary.
    '''

    mask = get_text_field_mask(input_tokens)
    embeddings = self.embedder(input_tokens)
    rnn_hidden = self.rnn(embeddings, mask)
    out_logits = self.hidden2out(rnn_hidden)
    loss = sequence_cross_entropy_with_logits(out_logits, output_tokens['tokens'], mask)

    return {'loss': loss}

the out_logits variable contains probabilities of tokens, how to dispaly these tokens.
the outlogits gives :
 array([[ 0.02416356,  0.0195566 , -0.03279119,  0.057118  ,  0.05091334,
    -0.01906729, -0.05311333,  0.04695245,  0.06872341,  0.05173637,
    -0.03523348, -0.00537474, -0.03946163, -0.05817827, -0.04316377,
    -0.06042208,  0.01190596,  0.00574979,  0.01183304,  0.02330608,
     0.04587644,  0.02319966,  0.0020873 ,  0.03781978, -0.03975108,
    -0.0131919 ,  0.00393738,  0.04785313,  0.00159995,  0.05751844,
     0.05420169, -0.01404533, -0.02716331, -0.03871592,  0.00949999,
    -0.02924301,  0.03504215,  0.00397302, -0.0305252 , -0.00228448,
     0.04034173,  0.01458408],
   [ 0.02050283,  0.0204745 , -0.03081856,  0.06295916,  0.04601778,
    -0.0167818 , -0.05653084,  0.05017883,  0.07212739,  0.06197165,
    -0.03590995, -0.01142827, -0.03807197, -0.05942211, -0.0375165 ,
    -0.06769539,  0.01200251,  0.01012686,  0.01514241,  0.01875677,
     0.04499928,  0.02748671,  0.0012517 ,  0.04062563, -0.04049949,
    -0.01986902,  0.00630998,  0.05092276,  0.00276728,  0.05341531,
     0.05047017, -0.01111878, -0.03038253, -0.04320357,  0.01768938,
    -0.03470382,  0.03567442,  0.00776757, -0.02703476, -0.00392571,
     0.04700187,  0.01671317]] dtype=float32)}

i want to convert the last array to token ?


title: Getting OSError: [WinError 193] %1 is not a valid Win32 application when trying to load Pytorch related dll

body:
When I try to use Pytorch I get the following error:
OSError: [WinError 193] %1 is not a valid Win32 application. Error loading "C:\Users\ANKUSH\AppData\Roaming\Python\Python38\site-packages\torch\lib\asmjit.dll" or one of its dependencies.

How can I solve this ?


title: pyTorch equivalent of SciPy.minmise

body:
Been solving a minimal likelihood problem in order to learn a reward function for an inverse reinforcement learning model. I've built it in NumPy and managed to minimise the likelihood function effectively using scipy.minimise with the gradient being returned by likelihood function.
But...
I've now integrated my implementation to pyTorch and scipy.minimise doesn't work all to great. It keeps terminating with a ABNORMAL_TERMINATION_IN_LNSRCH error.
I realise there is the torch.optim library and I tried to use the torch.optim.lbfgs() method but it seems to be built for neural nets and I can't figure out how to configure it for max/min likelihood problem.
Does anyone know to configure pyTorch.optim to match this likelihood minimisation using scipy.minimise:
#set likelihood variables
lh = Likelihood()
lh.set_variables_for_likehood(mdp_data, N, T, example_samples)

#minimise likelihood with analytically computed gradient
guess = torch.randn(mdp_params['n']**2,1)
res = minimize(lh.negated_likelihood, guess, jac=True, method="L-BFGS-B", options={'disp': True})

for the record my likelihood function is:
def negated_likelihood(self, r):

        if(torch.is_tensor(r) == False):
            r = torch.tensor(r)

        #Reshape R to expected
        if(r.shape != (4,5)):
            r = torch.reshape(r, (4,1))
            r = r.repeat((1, 5))

        #Solve MDP with current reward
        v, q, logp, p = linearvalueiteration(self.mdp_data, r)

        #Calculate likelihood from logp
        likelihood = sum(sum(logp*self.mu_sa))
        likelihood = likelihood.item() #get scalar value

        #Compute state visitation count D
        D = linearmdpfrequency(self.mdp_data,p,self.initD)

        #Compute gradient.
        dr = self.muE - torch.matmul(torch.t(self.F),D)
        dr = -dr #Invert for descent.
        dr = dr.detach().cpu().numpy() #convert to array

        return -(likelihood), dr


title: PyTorch: Vectorizing patch selection from a batch of images

body:
Suppose I have a batch of images as a tensor, for example:
images = torch.zeros(64, 3, 1024, 1024)

Now, I want to select a patch from each of those images. All the patches are of the same size, but have different starting positions for each image in the batch.
size_x = 100
size_y = 100
start_x = torch.zeros(64)
start_y = torch.zeros(64)

I can achieve the desired result like that:
result = []
for i in range(arr.shape[0]):
    result.append(arr[i, :, start_x[i]:start_x[i]+size_x, start_y[i]:start_y[i]+size_y])
result = torch.stack(result, dim=0)

The question is -- is it possible to do the same thing faster, without a loop? Perhaps there is some form of advanced indexing, or a PyTorch function that can do this?


title: Tensorflow to PyTorch

body:
I'm transffering a Tensorflow code to a PyTorch code.
Below lines are the problem I couldn't solve yet.
I'm not familiar with PyTorch so that it's not easy for me to find the matching methods in PyTorch library.
Anyone can help me?
p.s. The shape of alpha is (batch, N).
alpha_cumsum = tf.cumsum(alpha, axis = 1)
len_batch = tf.shape(alpha_cumsum)[0]
rand_prob = tf.random_uniform(shape = [len_batch, 1], minval = 0., maxval = 1.)
alpha_relu = tf.nn.relu(rand_prob - alpha_cumsum)
alpha_index = tf.count_nonzero(alpha_relu, 1)
alpha_hard  = tf.one_hot(alpha_index, len(a))


title: Custom PyTorch optimizer is not working properly and am unable to access gradients

body:
I'm trying to implement the elastic averaging stochastic gradient descent (EASGD) algorithm from the paper Deep Learning with Elastic Averaging SGD and was running into some trouble.
I'm using PyTorch's torch.optim.Optimizer class and referencing the official implementation of SGD and the official implementation of Accelerated SGD in order to start off somewhere.
The code that I have is:
import torch.optim as optim

class EASGD(optim.Optimizer):
    def __init__(self, params, lr, tau, alpha=0.001):
        self.alpha = alpha

        if lr < 0.0:
            raise ValueError(f"Invalid learning rate {lr}.")

        defaults = dict(lr=lr, alpha=alpha, tau=tau)
        super(EASGD, self).__init__(params, defaults)

    def __setstate__(self, state):
        super(EASGD, self).__setstate__(state)

    def step(self, closure=None):
        loss = None
        if closure is not None:
            with torch.enable_grad():
                loss = closure()

        for group in self.param_groups:
            tau = group['tau']

            for t, p in enumerate(group['params']):
                x_normal = p.clone()
                x_tilde = p.clone()

                if p.grad is None:
                    continue

                if t % tau == 0:
                    p = p - self.alpha * (x_normal - x_tilde)
                    x_tilde = x_tilde + self.alpha * (x_normal - x_tilde)

                d_p = p.grad.data
                p.data.add_(d_p, alpha=-group['lr'])

        return loss

When I run this code, I get the following error:

/home/user/github/test-repo/easgd.py:50: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the gradient for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations.

Reading this PyTorch Discussion helped understand what the difference between leaf and non-leaf variables are, but I'm not sure how I should fix my code to make it work properly.
Any tips on what to do or where to look are appreciated. Thanks.


title: What is the Problem in my Building Softmax from Scratch in Pytorch

body:
I read this post ans try to build softmax by myself. Here is the code
import torch
import torchvision
import torchvision.transforms as transforms
import matplotlib.pyplot as plt
import time
import sys
import numpy as np

#============================ get the dataset =========================

mnist_train = torchvision.datasets.FashionMNIST(root='~/Datasets/FashionMNIST', train=True, download=True, transform=transforms.ToTensor())
mnist_test = torchvision.datasets.FashionMNIST(root='~/Datasets/FashionMNIST', train=False, download=True, transform=transforms.ToTensor())

batch_size = 256
num_workers = 0

train_iter = torch.utils.data.DataLoader(mnist_train, batch_size=batch_size, shuffle=True, num_workers=num_workers)
test_iter = torch.utils.data.DataLoader(mnist_test, batch_size=batch_size, shuffle=False, num_workers=num_workers)

#============================     train      =========================
num_inputs = 28 * 28
num_outputs = 10
epochs = 5
lr = 0.05

# Initi the Weight and bia
W = torch.tensor(np.random.normal(0, 0.01, (num_inputs, num_outputs)), dtype=torch.float)
b = torch.zeros(num_outputs, dtype=torch.float)
W.requires_grad_(requires_grad = True)
b.requires_grad_(requires_grad=True)

# softmax function
def softmax(X):
    X = X.exp()
    den = X.sum(dim=1, keepdim=True)
    return X / den

# loss
def cross_entropy(y_hat, y):
    return - torch.log(y_hat.gather(1, y.view(-1, 1))).sum()

# accuracy function
def accuracy(y_hat, y):
    return (y_hat.argmax(dim=1) == y).float().mean().item()

for epoch in range(epochs):

    train_loss_sum = 0.0
    train_acc_sum = 0.0
    n_train = 0

    for X, y in train_iter:
        # X.shape: [256, 1, 28, 28]
        # y.shape: [256]

        # flatten the X into [256, 28*28]
        X = X.flatten(start_dim=1)
        y_pred = softmax(torch.mm(X, W) + b)

        loss = cross_entropy(y_pred, y)

        loss.backward()

        W.data = W.data - lr * W.grad
        b.data = b.data - lr* b.grad

        W.grad.zero_()
        b.grad.zero_()

        train_loss_sum += loss.item()

        train_acc_sum += accuracy(y_pred, y)
        n_train += y.shape[0]

    # evaluate the Test

    test_acc, n_test = 0.0, 0
    with torch.no_grad():

        for X_test, y_test in test_iter:
            X_test = X_test.flatten(start_dim=1)
            y_test_pred = softmax(torch.mm(X_test, W) + b)
            test_acc += accuracy(y_test_pred, y_test)
            n_test += y_test.shape[0]

    print('epoch %d, loss %.4f, train acc %.3f, test acc %.3f'
              % (epoch + 1, train_loss_sum/n_train , train_acc_sum / n_train, test_acc / n_test))

Compare with original post, Here I turn
def cross_entropy(y_hat, y):
    return - torch.log(y_hat.gather(1, y.view(-1, 1)))

into
def cross_entropy(y_hat, y):
    return - torch.log(y_hat.gather(1, y.view(-1, 1))).sum()

Since the backward need a scalar.
However, My results are
epoch 1, loss nan, train acc 0.000, test acc 0.000
epoch 2, loss nan, train acc 0.000, test acc 0.000
epoch 3, loss nan, train acc 0.000, test acc 0.000
epoch 4, loss nan, train acc 0.000, test acc 0.000
epoch 5, loss nan, train acc 0.000, test acc 0.000

Any idea?
Thanks.


title: How to confirm PyTorch runtime CUDA path？

body:
I create a PyTorch environment with conda.
conda install pytorch==1.7.0 torchvision torchaudio cudatoolkit=11.0 -c pytorch

but in the OS, there is already a CUDA in /usr/local/cuda-10.1
and a /usr/local/cuda point to /usr/local/cuda-10.1
There are two methods to show cuda version:

torch.version.cuda
torch.utils.cpp_extension.CUDA_HOME

I want to know which one has the runtime that exceeds the calculation?


title: Sorting a Pytorch Tensor by Trace

body:
I have a (100,64,22,3,3) shaped pytorch tensor, and I would like to sort along axis=0 by the trace of the (3,3) components. The code I have below works, but it is very slow due to the for loops. Is there a way to vectorize the operation to speed it up?
x=torch.rand(100,64,22,3,3)
x_sorted=torch.zeros((x.shape[0],x.shape[1],x.shape[2],x.shape[3],x.shape[4]))
            for i in range(x.shape[0]):
              #compute tensorized trace
              trace=new=torch.diagonal(x[i], dim1=-2, dim2=-1).sum(-1)
              #Sort the trace
              trace_values,trace_ind=torch.sort(trace,dim=0,descending=True)
              for j in range(x_sorted.shape[1]):
                for k in range(x_sorted.shape[2]):
                  x_sorted[i,j,k]=x[i,trace_ind[j,k],k]


title: How can I visualize similarity scores in a color schemes? (Tensor)

body:
I have a similarity scores in "sims" variable and I want to plotting similarity scores in a matrix preferably in color schemes. The more information about the "sims" variable is like that:
  import tensorflow.compat.v2 as tf

  print(sims) #Tensor("ExpandDims:0", shape=(1, 64, 64, 1), dtype=float32)
  print(sims[0]) #Tensor("strided_slice_6:0", shape=(64, 64, 1), dtype=float32)
  tf.print(sims[0][:], output_stream=sys.stderr)

The output of these rows are given the following screen shot.

To visualize the scores in color matrix I try to use plt.matshow method but I cannot convert the Tensor to numpy array (sims[0].numpy()) because it gives an error.

AttributeError: 'Tensor' object has no attribute 'numpy'.

Also I can't use plt.matshow. It gives the following error:

NotImplementedError: Cannot convert a symbolic Tensor
(strided_slice_10:0) to a numpy array. This error may indicate that
you're trying to pass a Tensor to a NumPy call, which is not supported

How can I visualize these scores in a matrix (64x64) like this?


title: How to generate sentences from nn.Transformer?

body:
I'm trying to make Transformer model to generate sentence. Now I'm trying to generate sentence by using beam search.
But my model generates only one sentence .
This symbol means this is the start of sentnece.
I have no idea why it is generated? Do you have good idea to solve it?
This is the code of Transformer model.
import math
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn import TransformerEncoder, TransformerEncoderLayer, TransformerDecoder, TransformerDecoderLayer

class TransformerModel(nn.Module):

    def __init__(self, in_token, out_token, ninp, nhead, nhid, nlayers, dropout=0.5):
        super(TransformerModel, self).__init__()
        self.model_type = 'Transformer'
        self.pos_encoder = PositionalEncoding(ninp, dropout)
        encoder_layers = TransformerEncoderLayer(ninp, nhead, nhid, dropout)
        self.transformer_encoder = TransformerEncoder(encoder_layers, nlayers)
        self.encoder = nn.Embedding(in_token, ninp)
        self.decoder = nn.Embedding(out_token, ninp)
        self.ninp = ninp
        self.linear = nn.Linear(ninp, out_token)
        decoder_layers = TransformerDecoderLayer(ninp, nhead, nhid, dropout)
        self.transformer_decoder = TransformerDecoder(decoder_layers, nlayers, norm = self.linear)

        self.init_weights()

    def generate_square_subsequent_mask(self, sz):
        mask = (torch.triu(torch.ones(sz, sz)) == 1).transpose(0, 1)
        mask = mask.float().masked_fill(mask == 0, float('-inf')).masked_fill(mask == 1, float(0.0))
        return mask

    def init_weights(self):
        initrange = 0.1
        self.encoder.weight.data.uniform_(-initrange, initrange)
        # self.decoder.bias.data.zero_()
        self.decoder.weight.data.uniform_(-initrange, initrange)

    def forward(self, src, trg):

        src_mask = model.generate_square_subsequent_mask(src.size()[0]).to(device)
        trg_mask = model.generate_square_subsequent_mask(trg.size()[0]).to(device)
        src = self.encoder(src)
        trg = self.decoder(trg)
        src = self.pos_encoder(src)
        trg = self.pos_encoder(trg)
        output = self.transformer_encoder(src, mask = src_mask)
        output = self.transformer_decoder(trg, output,tgt_mask = trg_mask)

        return output

class PositionalEncoding(nn.Module):

    def __init__(self, d_model, dropout=0.1, max_len=5000):
        super(PositionalEncoding, self).__init__()
        self.dropout = nn.Dropout(p=dropout)

        pe = torch.zeros(max_len, d_model)
        position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
        div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model))
        #print(pe[:, 0::2].size())
        #print(pe[:, 1::2].size())
        #print((position*div_term).size())
        pe[:, 0::2] = torch.sin(position * div_term)
        pe[:, 1::2] = torch.cos(position * div_term)
        pe = pe.unsqueeze(0).transpose(0, 1)
        self.register_buffer('pe', pe)

    def forward(self, x):
        x = x + self.pe[:x.size(0), :]
        return self.dropout(x)

# this is the parametor of model

n_ntokens = len(SRC.vocab.stoi) # the size of vocabulary
out_ntokens = len(TRG.vocab.stoi)
emsize = 512 # embedding dimension
nhid = 512 # the dimension of the feedforward network model in nn.TransformerEncoder
nlayers = 1 # the number of nn.TransformerEncoderLayer in nn.TransformerEncoder
nhead = 2 # the number of heads in the multiheadattention models
dropout = 0.3 # the dropout value
model = TransformerModel(in_ntokens, out_ntokens, emsize, nhead, nhid, nlayers, dropout).to(device)

And this is the code to generate sentence.
def gen_sentence(sentence, src_field, trg_field, model, max_len = 50):
    model.eval()

    tokens = [src_field.init_token] + \
        tokenizer(sentence) + [src_field.eos_token]
    src = [src_field.vocab.stoi[i] for i in tokens]
    src = torch.LongTensor([src])
    src = torch.t(src)
    src = src.to(device)

    src_tensor = model.encoder(src)
    src_tensor = model.pos_encoder(src_tensor).to(device)
    with torch.no_grad():
        src_output = model.transformer_encoder(src_tensor)#, mask = src_mask)

    trg = trg_field.vocab.stoi[trg_field.init_token]
    trg = torch.LongTensor([[trg]]).to(device)
    output = []
    for i in range(max_len):
        trg_tensor = model.decoder(trg)
        trg_tensor = model.pos_encoder(trg_tensor).to(device)
        trg_mask = model.generate_square_subsequent_mask(trg_tensor.size()[0]).to(device)
        with torch.no_grad():
            pred = model.transformer_decoder(trg_tensor, src_output, tgt_mask = trg_mask)
        pred_word_index = pred.argmax(2)[-1]
        output.append(pred_word_index)
        if pred_word_index == trg_field.vocab.stoi[trg_field.eos_token]:
            break

        last_index = torch.LongTensor([[pred_word_index.item()]]).to(device)
        trg = torch.cat((trg, last_index))

    # predict = "".join(output)
    predict = [trg_field.vocab.itos[i] for i in output]
    predict = "".join(predict)
    # print(predict)

    return predict

# this is the code when I use this function
sentence = "あまりにも気温が高いということでそれをやめてですね電車で来ました "
gen_sentence(sentence, SRC, TRG, model)

this is the output.
<sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos><sos>


title: Calculate alpha values with torch mean?

body:
I'm trying to calculate the alpha values as explained here.
I have as argument a tensor with shape (1, 512, 14, 14). To calculate alpha values I need to calculate the average of all dimensions except the channel dimension, so the output will have the shape (1, k, 1, 1) which is essentialy (k,).
How can I do this in PyTorch?
Thanks!


title: max operation for selective elements, not for all

body:
I am coding PyTorch. Between the torch inference code, I add some peripheral code for my own interest. This code works fine, but it is too slow. The reason might be for iteration. So, i need parallel and fast way of doing this.
It is okay to do this in tensor, Numpy, or just python array.
I made a function named selective_max to find maximum value in arrays. But the problem is that I don't want a maximum among the whole arrays, but among specific candidates which is designated by mask array. Let me show the gist of this function (below shows the code itself)
Input
x [batch_size , dim, num_points, k] : x is a original input, but this becomes [batch_size, num_points, dim, k] by x.permute(0,2,1,3).
batch_size is a well-known definition in the deep learning society. In every mini batch, there are many points. And a single point is represented by dim length feature. For each feature element, there are k potential candidates which is target of max function later.
mask [batch_size, num_points, k] : This array is similar to x without dim. Its element is either 0 or 1. So, I use this as a mask signal, like do max operation only on 1 masked value.
Kindly see the code below with this explanation. I use 3 for iteration. Let's say we target a specific batch and a specific point. For a specific batch and a specific point, x has [dim, k] array. And mask has [k] array which consists of either 0 or 1. So, I extract the non-zero index from [k] array and use this for extracting specific elements in x dim by dim('for k in range(dim)').
Toy example
Let's say we are in the second for iteration. So, we now have [dim, k] for x and [k] for mask. For this toy example, i presume k=3 and dim=4. x = [[3,2,1],[5,6,4],[9,8,7],[12,11,10]], k=[0,1,1]. So, output would be [2,6,8,11], not [3, 6, 9, 12].
Previous attempt
I try { mask.repeat(0,0,1,0) *(element-wise mul) x } and do the max operation. But, '0' might the max value, because the x might have minus values in all array. So, this would result in wrong operation.
def selective_max2(x, mask): # x : [batch_size , dim, num_points, k] , mask : [batch_size, num_points, k]
batch_size = x.size(0)
dim = x.size(1)
num_points = x.size(2)
k = x.size(3)
device = torch.device('cuda')

x = x.permute(0,2,1,3) # : [batch, num_points, dim, k]
#print('permuted x dimension : ',x.size())

x = x.detach().cpu().numpy()
mask = mask.cpu().numpy()
output = np.zeros((batch_size,num_points,dim))

for i in range(batch_size):
 for j in range(num_points):
  query=np.nonzero(mask[i][j]) # among mask entries, we get the index of nonzero values.
  for k in range(dim): # for different k values, we get the max value.
   # query is index of nonzero values. so, using query, we can get the values that we want.
   output[i][j][k] = np.max(x[i][j][k][query])

output = torch.from_numpy(output).float().to(device=device)
output = output.permute(0,2,1).contiguous()
return output


title: Pytorch Dataloader - How to to train with various train & val dataset sizes (10k Samples, 20k... 100k samples) to compare dataset size impact?

body:
i want to compare model performances depending on various sizes of the training and validation dataset.
I'd like to be able to build a loop which trains my pytorch model using 10k train and 1k val data and linearly increase the dataset sizes until 100k train and 10k val dataset sizes.
I also need to take target classes into consideration, therefore implement stratified splits somehow. I don't know where and how to implement this.
Does anyone have an idea how to easly implent this into my dataset class defintion?
Can i implement it here somehow?
def __len__(self):
        return len(self.inputFolderDataset.imgs)

I thought of additionally defining an array "sizes = [10000,20000...100000]" to iterature through.
Thank you in advance!


title: TypeError: 'module' object is not callable self.transform

body:
I'm trying to iterate on a custom dataset, and it's failed on image transform.
transform = transforms.Compose([
    transforms.ToPILImage(),
    transforms.Resize((255, 255)),
    # transforms.PILToTensor()])
    transforms.ToTensor(),
    transforms.Normalize(mean_img, std_img),
])

class img_dataset_fun(Dataset):
    def __init__(self, csv_file, transform):
        self.csv_file = pd.read_csv(csv_file)
        self.transform = transform

    def __len__(self):
        return len(self.csv_file)

    def __getitem__(self, index):
        if torch.is_tensor(index):
            index = index.tolist()
        img_path = self.csv_file.iloc[index, 1]
        image = io.imread(img_path)
        if self.transform is not None:
            image = self.transform(image)

        return image

train_img_dataset = img_dataset_fun(csv_file="data.csv", transform=transform)
train_img_loader = torch.utils.data.DataLoader(
    train_img_dataset,
    batch_size=1,
    num_workers=0,
    shuffle=False,
)
it = iter(train_img_loader)
images_iter = next(it)

images_iter fails with the error:
TypeError                                 Traceback (most recent call last)
<ipython-input-345-31cbe584d6de> in <module>()
      7                     shuffle=False,)
      8 it = iter(train_img_loader)
----> 9 images_iter =next(it)

4 frames
<ipython-input-335-55277e02141a> in __getitem__(self, index)
     17         if self.transform is not None:
---> 18           image=self.transform(image)
     19
     20

TypeError: 'module' object is not callable

Any idea what might be the problem?


title: Train loss is decreasing, but accuracy remain the same

body:
this is the train and development cell for multi-label classification task using Roberta (BERT). the first part is training and second part is development (validation). train_dataloader is my train dataset and dev_dataloader is development dataset. my question is: why train loss is decreasing step by step, but accuracy doesn't increase so much? practically, accuracy is increasing until iterate 4, but train loss is decreasing until the last epoch (iterate). is this ok or there should be a problem?
train_loss_set = []
iterate = 4
for _ in trange(iterate, desc="Iterate"):
  model.train()

  train_loss = 0
  nu_train_examples, nu_train_steps = 0, 0

  for step, batch in enumerate(train_dataloader):
    batch = tuple(t.to(device) for t in batch)
    batch_input_ids, batch_input_mask, batch_labels = batch
    optimizer.zero_grad()
    output = model(batch_input_ids, attention_mask=batch_input_mask)
    logits = output[0]
    loss_function = BCEWithLogitsLoss()
    loss = loss_function(logits.view(-1,num_labels),batch_labels.type_as(logits).view(-1,num_labels))
    train_loss_set.append(loss.item())
    loss.backward()
    optimizer.step()
    train_loss += loss.item()
    nu_train_examples += batch_input_ids.size(0)
    nu_train_steps += 1

  print("Train loss: {}".format(train_loss/nu_train_steps))

###############################################################################

  model.eval()
  logits_pred,true_labels,pred_labels,tokenized_texts = [],[],[],[]

  # Predict
  for i, batch in enumerate(dev_dataloader):
    batch = tuple(t.to(device) for t in batch)
    batch_input_ids, batch_input_mask, batch_labels = batch
    with torch.no_grad():
      out = model(batch_input_ids, attention_mask=batch_input_mask)
      batch_logit_pred = out[0]
      pred_label = torch.sigmoid(batch_logit_pred)
      batch_logit_pred = batch_logit_pred.detach().cpu().numpy()
      pred_label = pred_label.to('cpu').numpy()
      batch_labels = batch_labels.to('cpu').numpy()

    tokenized_texts.append(batch_input_ids)
    logits_pred.append(batch_logit_pred)
    true_labels.append(batch_labels)
    pred_labels.append(pred_label)

  pred_labels = [item for sublist in pred_labels for item in sublist]
  true_labels = [item for sublist in true_labels for item in sublist]
  threshold = 0.4
  pred_bools = [pl>threshold for pl in pred_labels]
  true_bools = [tl==1 for tl in true_labels]

  print("Accuracy is: ", jaccard_score(true_bools,pred_bools,average='samples'))
torch.save(model.state_dict(), 'bert_model')

and the outputs:
Iterate:   0%|          | 0/10 [00:00<?, ?it/s]

Train loss: 0.4024542534684801

/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1272: UndefinedMetricWarning: Jaccard is ill-defined and being set to 0.0 in samples with no true or predicted labels. Use `zero_division` parameter to control this behavior.
  _warn_prf(average, modifier, msg_start, len(result))

Accuracy is:  0.5806403013182674

Iterate:  10%|█         | 1/10 [03:21<30:14, 201.64s/it]

Train loss: 0.2972540049911379
Accuracy is:  0.6091337099811676

Iterate:  20%|██        | 2/10 [06:49<27:07, 203.49s/it]

Train loss: 0.26178574864264137
Accuracy is:  0.608361581920904

Iterate:  30%|███       | 3/10 [10:17<23:53, 204.78s/it]

Train loss: 0.23612180122962365
Accuracy is:  0.6096717783158462

Iterate:  40%|████      | 4/10 [13:44<20:33, 205.66s/it]

Train loss: 0.21416303515434265
Accuracy is:  0.6046892655367231

Iterate:  50%|█████     | 5/10 [17:12<17:11, 206.27s/it]

Train loss: 0.1929110718982203
Accuracy is:  0.6030885122410546

Iterate:  60%|██████    | 6/10 [20:40<13:46, 206.74s/it]

Train loss: 0.17280191068465894
Accuracy is:  0.6003766478342749

Iterate:  70%|███████   | 7/10 [24:08<10:21, 207.04s/it]

Train loss: 0.1517329115446631
Accuracy is:  0.5864783427495291

Iterate:  80%|████████  | 8/10 [27:35<06:54, 207.23s/it]

Train loss: 0.12957811209705325
Accuracy is:  0.5818832391713747

Iterate:  90%|█████████ | 9/10 [31:03<03:27, 207.39s/it]

Train loss: 0.11256680189521162
Accuracy is:  0.5796045197740114

Iterate: 100%|██████████| 10/10 [34:31<00:00, 207.14s/it]


title: Why does nn.Conv1d work on 2d feature [b, c, h, w]?

body:
I am wondering why conv1d works on 2d feature(batch, channel, height, width).
An nn.Conv1d(channel, channel, kernel_size=(1,1)) works when I put 2d feature, but gives different result from nn.Conv2d(channel, channel, kernel_size=1).
I want to know why conv1d works and what it mean by 2d kernel size in 1d convolution.


title: How to update classification layers without changing weights for convolutional layers

body:
I have a CNN with numerous convolutional layers. To each of these convolutional layers I have attached a classifier, to check outputs of intermediary layers. After losses have been produced for each of these classifiers, I want to update the weights for the classifier without touching the weights for the convolutional layers. This code:
for i in range(len(loss_per_layer)):
    loss_per_layer[i].backward(retain_graph=True)
    self.classifiers[i].weight.data -= self.learning_rate * self.alpha[i] * self.classifiers[i].weight.grad.data
    self.classifiers[i].bias.data -= self.learning_rate * self.alpha[i] * self.classifiers[i].bias.grad.data

allows me to do so if the classifier consists of a singular nn.Linear layer. However, my classifiers are of the shape:
self.classifiers.append(nn.Sequential(
    nn.Linear(int(feature_map * input_shape[1] * input_shape[2]), 100),
    nn.ReLU(True),
    nn.Dropout(),
    nn.Linear(100, self.num_classes),
    nn.Sigmoid(),
    ))

How can I update the weights of the Sequential block without touching the rest of the network? I have recently changed from keras to pytorch and so am unsure on how exactly to utilize the optimizer.step() function for this situation, but I have a suspicion it can be done using that.
Please note, I need a generic solution for a Sequential block of any shape, as it will change in future iterations of the model.
Any help is much appreciated.


title: Python re-write jupyter notebook to python script problem

body:
I have a very simple RNN model which I can execute on Jupyter notebook without any problem, now I want to embed this model in my Django project, so I downloaded the .ipynb file then saved it as .py file, then I just put it in my Django project.
However, the first problem is VSCode says I have unresolved import 'MLutils, the MLutils is my utility file in the same folder.
The second problem is if I just run this file, I'll get this error RuntimeError: cannot perform reduction function argmax on a tensor with no elements because the operation does not have an identity
but if I use the Run Cell Above button of the VSCode, I'll get the correct result.
The code I want to run is this, called MLTrain.py
#!/usr/bin/env python
# coding: utf-8

# In[1]:

import torch
import torch.nn as nn
import matplotlib.pyplot as plt

from MLutils import ALL_LETTERS, N_LETTERS
from MLutils import load_data, letter_to_tensor, line_to_tensor, random_training_example

class RNN(nn.Module):
    # implement RNN from scratch rather than using nn.RNN
    # # number of possible letters, hidden size, categories number
    def __init__(self, input_size, hidden_size, output_size):
        super(RNN, self).__init__()

        self.hidden_size = hidden_size
        #Define 2 different liner layers
        self.i2h = nn.Linear(input_size + hidden_size, hidden_size)
        self.i2o = nn.Linear(input_size + hidden_size, output_size)
        self.softmax = nn.LogSoftmax(dim=1)

    def forward(self, input_tensor, hidden_tensor):
        # combine input and hidden tensor
        combined = torch.cat((input_tensor, hidden_tensor), 1)
        # apply the Linear layers and the softmax
        hidden = self.i2h(combined)
        output = self.i2o(combined)
        output = self.softmax(output)
        # return 2 different tensors
        return output, hidden

    # need some initial hidden states in the begining.
    def init_hidden(self):
        return torch.zeros(1, self.hidden_size)
# dictionary with the country as the key and names as values
category_lines, all_categories = load_data()
# number of categories
n_categories = len(all_categories)

# a Hyperparameter
n_hidden = 128
# number of possible letters, hidden size, output size
rnn = RNN(N_LETTERS, n_hidden, n_categories)

# one step
input_tensor = letter_to_tensor('A')
hidden_tensor = rnn.init_hidden()

output, next_hidden = rnn(input_tensor, hidden_tensor)
#print(output.size())
#>>> size: [1,18]
#print(next_hidden.size())
#>>> size: [1,128]
# whole sequence/name
input_tensor = line_to_tensor('if')
hidden_tensor = rnn.init_hidden()

output, next_hidden = rnn(input_tensor[0], hidden_tensor)
print(output.size())
print(next_hidden.size())

# apply softmax in the end.
# this is the likelyhood of each character of each category
def category_from_output(output):
    # return index of the greatest value
    category_idx = torch.argmax(output).item()
    return all_categories[category_idx]
print(category_from_output(output))

criterion = nn.NLLLoss()
# hyperparameter
learning_rate = 0.005
optimizer = torch.optim.SGD(rnn.parameters(), lr=learning_rate)
# whole name as tensor,
def train(line_tensor, category_tensor):
    hidden = rnn.init_hidden()
    #line_tensor.size()[0]: the length of the name
    for i in range(line_tensor.size()[0]):
        # apply the current character and the previous hidden state.
        output, hidden = rnn(line_tensor[i], hidden)

    loss = criterion(output, category_tensor)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    return output, loss.item()

current_loss = 0
all_losses = []
plot_steps, print_steps = 100, 500
n_iters = 2000
for i in range(n_iters):
    category, line, category_tensor, line_tensor = random_training_example(category_lines, all_categories)

    output, loss = train(line_tensor, category_tensor)
    current_loss += loss

    if (i+1) % plot_steps == 0:
        all_losses.append(current_loss / plot_steps)
        current_loss = 0

    if (i+1) % print_steps == 0:
        guess = category_from_output(output)
        correct = "CORRECT" if guess == category else f"WRONG ({category})"
        print(f"{i+1} {(i+1)/n_iters*100} {loss:.4f} {line} / {guess} {correct}")

plt.figure()
plt.plot(all_losses)
plt.show()
# model can be saved

def predict(input_line):
    print(f"\n> {input_line}")
    with torch.no_grad():
        line_tensor = line_to_tensor(input_line)

        hidden = rnn.init_hidden()

        for i in range(line_tensor.size()[0]):
            output, hidden = rnn(line_tensor[i], hidden)

        guess = category_from_output(output)
        print(guess)

if __name__ == "__main__":
    predict("abcde 1 ifelse")

# In[ ]:

The MLutils.py is this
import io
import os
import unicodedata
import string
import glob

import torch
import random

# alphabet small + capital letters + " .,;'"
ALL_LETTERS = string.ascii_letters + " .,;'"
N_LETTERS = len(ALL_LETTERS)

# Turn a Unicode string to plain ASCII, thanks to https://stackoverflow.com/a/518232/2809427
def unicode_to_ascii(s):
    return ''.join(
        c for c in unicodedata.normalize('NFD', s)
        if unicodedata.category(c) != 'Mn'
        and c in ALL_LETTERS
    )

def load_data():
    # Build the category_lines dictionary, a list of names per language
    category_lines = {}
    all_categories = []

    def find_files(path):
        return glob.glob(path)

    # Read a file and split into lines
    def read_lines(filename):
        lines = io.open(filename, encoding='utf-8').read().strip().split('\n')
        return [unicode_to_ascii(line) for line in lines]

    for filename in find_files('data/categories/*.txt'):
        category = os.path.splitext(os.path.basename(filename))[0]
        all_categories.append(category)

        lines = read_lines(filename)
        category_lines[category] = lines

    return category_lines, all_categories

# Find letter index from all_letters, e.g. "a" = 0
def letter_to_index(letter):
    return ALL_LETTERS.find(letter)

# Just for demonstration, turn a letter into a <1 x n_letters> Tensor
def letter_to_tensor(letter):
    tensor = torch.zeros(1, N_LETTERS)
    tensor[0][letter_to_index(letter)] = 1
    return tensor

# Turn a line into a <line_length x 1 x n_letters>,
# or an array of one-hot letter vectors
def line_to_tensor(line):
    tensor = torch.zeros(len(line), 1, N_LETTERS)
    for i, letter in enumerate(line):
        tensor[i][0][letter_to_index(letter)] = 1
    return tensor

def random_training_example(category_lines, all_categories):

    def random_choice(a):
        random_idx = random.randint(0, len(a) - 1)
        return a[random_idx]

    category = random_choice(all_categories)
    line = random_choice(category_lines[category])
    category_tensor = torch.tensor([all_categories.index(category)], dtype=torch.long)
    line_tensor = line_to_tensor(line)
    return category, line, category_tensor, line_tensor

if __name__ == '__main__':
    print(ALL_LETTERS)
    print(unicode_to_ascii('Ślusàrski'))

    # category_lines, all_categories = load_data()
    # print(category_lines['Flow'][:5])

    # print(letter_to_tensor('J')) # [1, 57]
    # print(line_to_tensor('Jones').size()) # [5, 1, 57]

The Run Above Button is this

If I comment out the print here, I'll get this error

Error Full-Text
Traceback (most recent call last):
  File "c:\Users\Leslie\Desktop\todo_drf\MLModel\MLTrain.py", line 106, in <module>
    category, line, category_tensor, line_tensor = random_training_example(category_lines, all_categories)
  File "c:\Users\Leslie\Desktop\todo_drf\MLModel\MLutils.py", line 84, in random_training_example
    category = random_choice(all_categories)
  File "c:\Users\Leslie\Desktop\todo_drf\MLModel\MLutils.py", line 81, in random_choice
    random_idx = random.randint(0, len(a) - 1)
  File "C:\Users\Leslie\AppData\Local\Programs\Python\Python39\lib\random.py", line 338, in randint
    return self.randrange(a, b+1)
  File "C:\Users\Leslie\AppData\Local\Programs\Python\Python39\lib\random.py", line 316, in randrange
    raise ValueError("empty range for randrange() (%d, %d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (0, 0, 0)

That's really wired or I am really stupid on somewhere.
Someone please help me!
Many thanks.


title: Why is my parameter not changing and its gradient 0?

body:
I am building a really simple model to learn the parameter in a Poisson model and I am not sure where I am going wrong. I am using pytorch.nn and doing the following.
I made some really simple fake data
# This is the value I am trying to estimate

x = torch.tensor(2.0)

# This is a value drawn from the Poisson(x) distribution
# In this example it is 4

y = torch.poisson(x).reshape(1)

Then I just set up a really simple model
# I initialised the parameter that is going to estimate x with a random value (0.2)
# and set that it requires a gradient

a = torch.tensor([0.2], requires_grad = True)

# I define the loss function with log_input set to false

loss_function = torch.nn.PoissonNLLLoss(log_input = False)

# Defined the model

def model(a):
    return torch.poisson(a)

# And the parameter to be optimised
# I chose SGD arbitrarily, maybe this is the problem?

optimizer = torch.optim.SGD([a], lr = 0.1)

Then I do iterations to update a
for i in range(2000):

    # Forward pass

    y_pred = model(a)

    # Compute the loss

    loss = loss_function(y_pred, y)

    # Backprop

    optimizer.zero_grad()

    loss.backward()

    # Update parameters

    optimizer.step()

The problem is after this the a is still 0.2 and if I call a.grad it is 0. Where am I going wrong?
Thanks in advance
UPDATE
I have tried instead to initiate a class for the model inheriting a nn.Module. However the same problem persists :
class learning_model(nn.Module):

    def __init__(self):
        super().__init__()
        self.a = nn.Parameter(torch.rand(1))
        self.a.requires_grad = True

    def forward(self):
        return torch.poisson(self.a)

model = learning_model()

loss_function = nn.PoissonNLLLoss(log_input = False)

optimizer = torch.optim.SGD(model.parameters(), lr = 0.1)

print(model.a)

Outputs:
Parameter containing:
tensor([0.1402], requires_grad=True)

Then:
for i in range(20):

    # Forward pass
    y_pred = model()

    # Compute the loss
    loss = loss_function(y_pred, y)

    # Backprop
    optimizer.zero_grad()

    loss.backward()

    # Update parameters
    optimizer.step()

print(model.a, '\n gradient:', model.a.grad)

Outputs:
Parameter containing:
tensor([0.1402], requires_grad=True)
 gradient: tensor([0.])


title: torch: num_workers param in Dataloader provokes an error

body:
I want to learn my model using GPU, but I'm getting an error when i try to point number of threads in Dataloader in PyTorch. I do:
X_train, X_test, y_train, y_test = model_selection.train_test_split(X_reduced,
                                                                    y,
                                                                    test_size=0.4)
X_test, X_val, y_test, y_val = model_selection.train_test_split(X_test,
                                                                y_test,
                                                                test_size=0.5)

dtype = config['dtype']

train_dataset = torch.utils.data.TensorDataset(torch.tensor(X_train, dtype=dtype, requires_grad=True),
                                               torch.tensor(y_train.values, dtype=dtype, requires_grad=True))
valid_dataset = torch.utils.data.TensorDataset(torch.tensor(X_val, dtype=dtype, requires_grad=True),
                                               torch.tensor(y_val.values, dtype=dtype, requires_grad=True))
train_dataloader = torch.utils.data.DataLoader(train_dataset, shuffle=True, batch_size=config['batch_size'], num_workers=8)
valid_dataloader = torch.utils.data.DataLoader(valid_dataset, shuffle=True, batch_size=config['batch_size'], num_workers=8)

Also i'm using torchLightning:
class PostflopNet(pl.LightningModule):
    def __init__(self, n_inputs):
        super().__init__()

        self.encoder = torch.nn.Sequential(
        torch.nn.Linear(n_inputs, 512),
        torch.nn.ReLU(),
        torch.nn.Linear(512, 128),
        torch.nn.ReLU(),
        torch.nn.Linear(128, 32),
        torch.nn.ReLU(),
        torch.nn.Linear(32, 8),
        torch.nn.ReLU(),
        torch.nn.Linear(8, 1))

    def forward(self, x):
        embedding = self.encoder(x)
        return embedding

    def configure_optimizers(self):
        optimizer = torch.optim.Adam(self.parameters(), lr=1.0e-2)
        scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=50, gamma=0.6)
        return [optimizer], [scheduler]

    def training_step(self, train_batch, batch_idx):
        inputs, labels = train_batch
        predicted = self(inputs)
        loss = torch.nn.MSELoss().__call__(inputs, labels)
        print('train_loss: ', loss)
        print(inputs.shape)
        return loss

model = PostflopNet(X_train.shape[1])
trainer = pl.Trainer(max_epochs=config['n_train_epoch'], gpus=1)

trainer.fit(model, train_dataloader, valid_dataloader)

But i get this error:
File "/home/neighbourhood@netsrv.pw/.local/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 55, in default_collate
    return torch.stack(batch, 0, out=out)
RuntimeError: stack(): functions with out=... arguments don't support automatic differentiation, but one of the arguments requires grad.

If i remove num_worker param in Dataloader, it learns successfully.
What is the problem?


title: Is there a element-map function in pytorch?

body:
I'm new in PyTorch and I come from functional programming languages(where map function is used everywhere). The problem is that I have a tensor and I want to do some operations on each element of the tensor. The operation may be various so I need a function like this:
map : (Numeric -> Numeric) -> Tensor -> Tensor

e.g. map(lambda x: x if x < 255 else -1, tensor) # the example is simple, but the lambda may be very complex

Is there such a function in PyTorch? How should I implement such function?