title: How can I fix "python setup.py egg_info" failed pytorch install error?

body:
I am trying to install pytorch from github. I type in
    pip3 install git+pip3 install git+https://github.com/pytorch/pytorch.git

I am getting this error. I've tried upgrading pip, switch from python 2 to 3, etc.

 File "/Users/MYK/anaconda3/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '4']' returned non-zero exit status 2.
Building wheel torch-1.2.0a0+d4464d3
-- Building version 1.2.0a0+d4464d3
cmake -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=  -DCMAKE_C_FLAGS=  -DCMAKE_EXE_LINKER_FLAGS= -DCMAKE_INSTALL_PREFIX=/private/var/folders/xs/xqll03b96p74d6v15mwfvwgw0000gq/T/pip-req-build-ryxkksy0/torch -DCMAKE_PREFIX_PATH=/Users/MYK/anaconda3/lib/python3.6/site-packages -DCMAKE_SHARED_LINKER_FLAGS= -DINSTALL_TEST=True -DNUMPY_INCLUDE_DIR=/Users/MYK/anaconda3/lib/python3.6/site-packages/numpy/core/include -DPYTHON_EXECUTABLE=/Users/MYK/anaconda3/bin/python -DPYTHON_INCLUDE_DIR=/Users/MYK/anaconda3/include/python3.6m -DPYTHON_LIBRARY=/Users/MYK/anaconda3/lib/libpython3.6m.dylib -DTORCH_BUILD_VERSION=1.2.0a0+d4464d3 -DUSE_CUDA=False -DUSE_DISTRIBUTED=True -DUSE_FBGEMM=True -DUSE_NCCL=False -DUSE_NUMPY=True -DUSE_SYSTEM_EIGEN_INSTALL=OFF -DUSE_SYSTEM_NCCL=False /private/var/folders/xs/xqll03b96p74d6v15mwfvwgw0000gq/T/pip-req-build-ryxkksy0
cmake --build . --target install --config Release -- -j 4

Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/xs/xqll03b96p74d6v15mwfvwgw0000gq/T/pip-req-build-ryxkksy0/


title: Can I get list of classes from `torchvision.datasets.ImageNet` without downloading dataset?

body: It looks like until recently there were no such list in pytorch (see here). Now we have ImageNet class in torchvision but it's not quite clear how can I get list of classes.


title: Why "conv1d" is different in C code, python and pytorch

body:
I want to reproduce "Conv1D" results of pytorch in C code.

I tried to implement "Conv1D" using three methods (C code, Python, Pytorch), but the results are different. Only seven fraction digits are reasonable. Assuming there are multiple layers of conv1d in the structure, the fraction digits accuracy will gradually decrease.

According to everyone's recommend, I tried to change the C code type of input data to double but the result is still incorrect.
Have I done something wrong?

For example:

　　The output of Pytorch: 0.2380688339471817017

　　The output of Python: 0.2380688637495040894

　　The output of C code (float): 0.2380688637

　　The output of C code (double): 0.238068885344539680

Here is my current implementation

Input:

　　input dim. = 80, output dim. = 128, kernel size = 5

　　Pytorch: Conv1D_input.npy, Conv1D_weight.npy

　　Python: Conv1D_input.npy, Conv1D_weight.npy (same as Pytorch)

　　C code: Conv1D_input.txt, Conv1D_weight.txt (convert from Pytorch, IEEE 754 single precision)

　　Pytorch

import torch
import numpy as np
from torch import nn
from torch.autograd import Variable
import torch.nn.functional as F
import argparse
import sys
import io
import time
import os

class RNN(nn.Module):
    def __init__(self, input_size, hidden_size):
        super(RNN, self).__init__()
        self.input_size = input_size
        self.hidden_size = hidden_size

        self.c1 = nn.Conv1d(input_size, hidden_size, kernel_size = 5, bias=False)
        self.c1.weight = torch.nn.Parameter(torch.Tensor(np.load("CONV1D_WEIGHT.npy")))

    def forward(self, inputs):
        c = self.c1(inputs)
        return c

input_size = 80
hidden_size = 128
kernel_size = 5

rnn = RNN(input_size, hidden_size)

inputs = torch.nn.Parameter(torch.Tensor(np.load("CONV1D_IN.npy")))
print("inputs", inputs)
outputs = rnn(inputs)
sub_np456 = outputs[0].cpu().detach().numpy()
np.savetxt("Pytorch_CONV1D_OUTPUT.txt", sub_np456)
print('outputs', outputs)

　　Python

import struct
import numpy as np

if __name__ == "__main__":
    row = 80
    col = 327
    count = 0
    res_out_dim = 128
    in_dim = 80
    kernel_size = 5
    filter = np.zeros((80, 5), dtype = np.float32)
    featureMaps = np.zeros((128, 323), dtype = np.float32)

    spectrum = np.load("CONV1D_INPUT.npy")
    weight = np.load("CONV1D_WEIGHT.npy")

    spectrum_2d = spectrum.reshape(80, 327)

    for i in range(res_out_dim):
        for j in range(in_dim):
            for k in range(kernel_size):
                filter[j][k] = weight[i][j][k]

        while count < (col-kernel_size+1):
            for j in range(in_dim):
                for k in range(count, kernel_size+count):
                    featureMaps[i][count] = featureMaps[i][count] + spectrum_2d[j][k]*filter[j][k-count]
            count = count + 1
        count = 0

    np.savetxt("Python_CONV1D_OUTPUT.txt", featureMaps)

　　C code (float)

#include<stdio.h>
#include<stdlib.h>
#include<math.h>
#include<time.h>

const char CONV1D_WEIGHT[] = "CONV1D_WEIGHT.txt";
const char CONV1D_INPUT[] = "CONV1D_INPUT.txt";

void parameterFree(float **matrix, int row)
{
    int i = 0;
    for(i=0; i<row; i++)
        free(matrix[i]);
    free(matrix);
}

float** createMatrix_2D(int row, int col)
{
    int i = 0;
    float **matrix = NULL;
    matrix = (float**)malloc(sizeof(float*) * row);
    if(matrix == NULL)
        printf("Matrix2D malloc failed\n");
    for(i=0; i<row; i++)
    {
        matrix[i] = (float*)malloc(sizeof(float) * col);
        if(matrix[i] == NULL)
            printf("Matrix2D malloc failed\n");
    }

    return matrix;
}

float** conv_1D(const char weightFile[], float **source, int *row, int *col, int in_dim, int res_out_dim, int kernel_size)
{
    float **filter = createMatrix_2D(in_dim, kernel_size);
    float **featureMaps = createMatrix_2D(res_out_dim, *col-kernel_size+1);
    int i = 0, j = 0, k = 0, count = 0;
    char str[10];
    float data = 0.0;
    FILE *fp = fopen(weightFile, "r");
    if(fp == NULL)
        printf("Resnet file open failed\n");
    else
    {
        /*initial featureMaps*/
        for(i=0; i<res_out_dim; i++)
        {
            for(j=0; j<*col-kernel_size+1; j++)
            {
                featureMaps[i][j] = 0.0;
            }
        }

        /*next filter*/
        for(i=0; i<res_out_dim; i++)
        {
            /*read filter*/
            for(j=0; j<in_dim; j++)
            {
                for(k=0; k<kernel_size; k++)
                {
                    fscanf(fp, "%s", str);
                    sscanf(str, "%x", &data);
                    filter[j][k] = data;
                }
            }

            /* (part of source * filter) */
            while(count < *col-kernel_size+1)
            {
                for(j=0; j<in_dim; j++)
                {
                    for(k=count; k<kernel_size+count; k++)
                    {
                        featureMaps[i][count] += source[j][k]*filter[j][k-count];
                    }
                }
                count++;
            }
            count = 0;
        }
        fclose(fp);
    }
    parameterFree(source, *row);
    parameterFree(filter, in_dim);
    *row = res_out_dim;
    *col = *col-kernel_size+1;

    return featureMaps;
}

int main()
{
    int row = 80;
    int col = 327;
    int in_dim = 80;
    int res_out_dim = 128;
    int kernel_size = 5;
    int i, j;
    float data;
    char str[10];

    float **input = createMatrix_2D(row, col);
    FILE *fp = fopen(CONV1D_INPUT, "r");
    FILE *fp2 = fopen("C code_CONV1D_OUTPUT.txt", "w");
    if(fp == NULL)
        printf("File open failed\n");
    else
    {
        for(i=0; i<row; i++)
        {
            for(j=0; j<col; j++)
            {
                fscanf(fp, "%s", str);
                sscanf(str, "%x", &data);
                input[i][j] = data;
            }
        }
    }

    float **CONV1D_ANS = conv_1D(CONV1D_WEIGHT, input, &row, &col, in_dim, res_out_dim, kernel_size);

    for(i=0; i<row; i++)
    {
        for(j=0; j<col; j++)
        {
            fprintf(fp2, "[%.12f] ", CONV1D_ANS[i][j]);
        }
        fprintf(fp2, "\n");
    }

    return 0;
}

　　C code (double)

#include<stdio.h>
#include<stdlib.h>
#include<math.h>
#include<time.h>

const char CONV1D_WEIGHT[] = "CONV1D_WEIGHT.txt";
const char CONV1D_INPUT[] = "CONV1D_INPUT.txt";

void parameterFree(double **matrix, int row)
{
    int i = 0;
    for(i=0; i<row; i++)
        free(matrix[i]);
    free(matrix);
}

double** createMatrix_2D(int row, int col)
{
    int i = 0;
    double **matrix = NULL;
    matrix = (double**)malloc(sizeof(double*) * row);
    if(matrix == NULL)
        printf("Matrix2D malloc failed\n");
    for(i=0; i<row; i++)
    {
        matrix[i] = (double*)malloc(sizeof(double) * col);
        if(matrix[i] == NULL)
            printf("Matrix2D malloc failed\n");
    }

    return matrix;
}

double** conv_1D(const char weightFile[], double **source, int *row, int *col, int in_dim, int res_out_dim, int kernel_size)
{
    double **filter = createMatrix_2D(in_dim, kernel_size);
    double **featureMaps = createMatrix_2D(res_out_dim, *col-kernel_size+1);
    int i = 0, j = 0, k = 0, count = 0;
    char str[10];
    float data = 0.0;
    FILE *fp = fopen(weightFile, "r");
    if(fp == NULL)
        printf("Resnet file open failed\n");
    else
    {
        /*initial featureMaps*/
        for(i=0; i<res_out_dim; i++)
        {
            for(j=0; j<*col-kernel_size+1; j++)
            {
                featureMaps[i][j] = 0.0;
            }
        }

        /*next filter*/
        for(i=0; i<res_out_dim; i++)
        {
            /*read filter*/
            for(j=0; j<in_dim; j++)
            {
                for(k=0; k<kernel_size; k++)
                {
                    fscanf(fp, "%s", str);
                    sscanf(str, "%x", &data);
                    filter[j][k] = (double)data;
                }
            }

            /* (part of source * filter) */
            while(count < *col-kernel_size+1)
            {
                for(j=0; j<in_dim; j++)
                {
                    for(k=count; k<kernel_size+count; k++)
                    {
                        featureMaps[i][count] += source[j][k]*filter[j][k-count];
                    }
                }
                count++;
            }
            count = 0;
        }
        fclose(fp);
    }
    parameterFree(source, *row);
    parameterFree(filter, in_dim);
    *row = res_out_dim;
    *col = *col-kernel_size+1;

    return featureMaps;
}

int main()
{
    int row = 80;
    int col = 327;
    int in_dim = 80;
    int res_out_dim = 128;
    int kernel_size = 5;
    int i, j;
    float data;
    char str[10];

    double **input = createMatrix_2D(row, col);
    FILE *fp = fopen(CONV1D_INPUT, "r");
    FILE *fp2 = fopen("C code_CONV1D_OUTPUT.txt", "w");
    if(fp == NULL)
        printf("File open failed\n");
    else
    {
        for(i=0; i<row; i++)
        {
            for(j=0; j<col; j++)
            {
                fscanf(fp, "%s", str);
                sscanf(str, "%x", &data);
                input[i][j] = (double)data;
            }
        }
    }

    double **CONV1D_ANS = conv_1D(CONV1D_WEIGHT, input, &row, &col, in_dim, res_out_dim, kernel_size);

    for(i=0; i<row; i++)
    {
        for(j=0; j<col; j++)
        {
            fprintf(fp2, "[%.18f] ", CONV1D_ANS[i][j]);
        }
        fprintf(fp2, "\n");
    }

    return 0;
}


title: Improving accuracy on a multi-class image classifiier

body:
I am building a classifier using the Food-101 dataset. The dataset has predefined training and test sets, both labeled. It has a total of 101,000 images. I’m trying to build a classifier model with >=90% accuracy for top-1. I’m currently sitting at 75%. The training set was provided unclean. But now, I would like to know some of the ways I can improve my model and what are some of the things I’m doing wrong.

I’ve partitioned the train and test images into their respective folders. Here, I am using 0.2 of the training dataset to validate the learner by running 5 epochs.

np.random.seed(42)
data = ImageList.from_folder(path).split_by_rand_pct(valid_pct=0.2).label_from_re(pat=file_parse).transform(size=224).databunch()

top_1 = partial(top_k_accuracy, k=1)
learn = cnn_learner(data, models.resnet50, metrics=[accuracy, top_1], callback_fns=ShowGraph)
learn.fit_one_cycle(5)

epoch   train_loss  valid_loss  accuracy    top_k_accuracy  time
0       2.153797    1.710803    0.563498    0.563498    19:26
1       1.677590    1.388702    0.637096    0.637096    18:29
2       1.385577    1.227448    0.678746    0.678746    18:36
3       1.154080    1.141590    0.700924    0.700924    18:34
4       1.003366    1.124750    0.707063    0.707063    18:25

And here, I’m trying to find the learning rate. Pretty standard to how it was in the lectures:

learn.lr_find()
learn.recorder.plot(suggestion=True)

LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 1.32E-06
Min loss divided by 10: 6.31E-08

Using the learning rate of 1e-06 to run another 5 epochs. Saving it as stage-2

learn.fit_one_cycle(5, max_lr=slice(1.e-06))
learn.save('stage-2')

epoch   train_loss  valid_loss  accuracy    top_k_accuracy  time
0       0.940980    1.124032    0.705809    0.705809    18:18
1       0.989123    1.122873    0.706337    0.706337    18:24
2       0.963596    1.121615    0.706733    0.706733    18:38
3       0.975916    1.121084    0.707195    0.707195    18:27
4       0.978523    1.123260    0.706403    0.706403    17:04

Previously I ran 3 stages altogether but the model wasn’t improving beyond 0.706403 so I didn’t want to repeat it. Below is my confusion matrix. I apologize for the terrible resolution. Its the doing of Colab.

Since I’ve created an additional validation set, I decided to use the test set to validate the saved model of stage-2 to see how well it was performing:

path = '/content/food-101/images'
data_test = ImageList.from_folder(path).split_by_folder(train='train', valid='test').label_from_re(file_parse).transform(size=224).databunch()

learn.load('stage-2')
learn.validate(data_test.valid_dl)

This is the result:

[0.87199837, tensor(0.7584), tensor(0.7584)]


title: Download the datasets from torchvision. Differences between the argument?

body:
For example it is written for the CelebA dataset:

torchvision.datasets.CelebA(root, split='train', download=False)

While for CIFAR there is writing:

torchvision.datasets.CIFAR10(root, train=True, download=False)

My question is: what is the difference between train=true and split=train?


title: Pytorch convert a pd.DataFrame which is variable length sequence to tensor

body:
I get a pandas DataFrame as follows and want to convert it to torch.tensor for embedding.
# output first 5 rows examples
print(df['col'].head(5))

                      col
0             [a, bc, cd]
1      [d, ed, fsd, g, h]
2  [i, hh, ihj, gfw, hah]
3                 [a, cb]
4                   [sad]

train_tensor = torch.from_numpy(train)

But it gets an error:

TypeError: can't convert np.ndarray of type numpy.str_. The only supported types are: float64, float32, float16, int64, int32, int16, int8, uint8, and bool.

It seems that from_numpy() doesn't support the variable lenght sequences.
So if want to initialize tensor form it what is the proper way?
And after getting the corresponding tensor I will try to add padding to variable length sequences and do embedding layer for it.
Could anyone help me?
Thanks in advances.


title: Python IO PyTorch DataLoader memory leak when opening images

body:
My code roughly looks like:
class MyDataset(torch.utils.data.Dataset):
  def __getitem__(self, idx):
    img = cv2.imread(file_paths[idx])
    return img

dataset = MyDataset()

loader = torch.utils.data.DataLoader(dataset, pin_memory=True, shuffle=True, num_workers=4)

for data in loader:
  pass

I run into memory issues before I can go through the whole dataset. The memory gradually gets eaten up until it hits the limit. It's something like 100,000 512x512 RGB images, but of course they shouldn't be staying in memory between loop iterations.
What's bizarre is that if I swap out file_paths[idx] for 'file_that_exists.jpg' I can run through the whole length of the dataset without the memory being gradually eaten up. So somehow memory seems to be allocated to unique files that are opened?
Note that this is not this issue: https://github.com/pytorch/pytorch/issues/13246 as it seems to happen regardless of num_workers and doesn't have anything to do with Python lists.


title: Should the parameters of the BatchNorm layer be regularized

body: When I use pytorch to train my CNN, the L2 regularization will be used to panalize the parameters in the model. But the pytorch code "weight decay" will use L2 to all the parameters which can be updated. Should the parameters in BatchNorm Layers be panalized by L2, too?


title: RuntimeError: Expected hidden[0] size (1, 1, 512), got (1, 128, 512) for LSTM pytorch

body:
I trained the LSTM with a batch size of 128 and during testing my batch size is 1, why do I get this error? I'm suppose to initialize the hidden size when doing testing?
Here is the code that i'm using, I initialize the hidden state init_hidden function as (number_of_layers, batch_size, hidden_size) since batch_first=True
class ImageLSTM(nn.Module):
def __init__(self,  n_inputs:int=49,
                    n_outputs:int=4096,
                    n_hidden:int=256,
                    n_layers:int=1,
                    bidirectional:bool=False):
    """
    Takes a 1D flatten images.
    """
    super(ImageLSTM, self).__init__()
    self.n_inputs   = n_inputs
    self.n_hidden   = n_hidden
    self.n_outputs  = n_outputs
    self.n_layers   = n_layers
    self.bidirectional = bidirectional
    self.lstm       = nn.LSTM(  input_size=self.n_inputs,
                                hidden_size=self.n_hidden,
                                num_layers=self.n_layers,
                                dropout  = 0.5 if self.n_layers>1 else 0,
                                bidirectional=self.bidirectional,
                                batch_first=True)
    if (self.bidirectional):
        self.FC         = nn.Sequential(
                                        nn.Linear(self.n_hidden*2, self.n_outputs),
                                        nn.Dropout(p=0.5),
                                        nn.Sigmoid()
                                            )

    else:
        self.FC         = nn.Sequential(
                                        nn.Linear(self.n_hidden, self.n_outputs),
                                        # nn.Dropout(p=0.5),
                                        nn.Sigmoid()
                                        )

def init_hidden(self, batch_size, device=None): # input 4D tensor: (batch size, channels, width, height)
    # initialize the hidden and cell state to zero
    # vectors:(number of layer, batch size, number of hidden nodes)
    if (self.bidirectional):
        h0 = torch.zeros(2*self.n_layers,  batch_size, self.n_hidden)
        c0 = torch.zeros(2*self.n_layers,  batch_size, self.n_hidden)
    else:
        h0 = torch.zeros(self.n_layers,  batch_size, self.n_hidden)
        c0 = torch.zeros(self.n_layers,  batch_size, self.n_hidden)

    if device is not None:
        h0 = h0.to(device)
        c0 = c0.to(device)
    self.hidden = (h0,c0)

def forward(self, X): # X: tensor of shape (batch_size, channels, width, height)
    # forward propagate LSTM
    lstm_out, self.hidden = self.lstm(X, self.hidden) # lstm_out: tensor of shape (batch_size, seq_length, hidden_size)
    # Decode the hidden state of the last time step
    out = self.FC(lstm_out[:, -1, :])
    return out


title: loading model failed in torchserving

body:
i learning to serve a model using pytorch serving and i am new to this serving
this is the handler file i created for serving the vgg16 model
i am using the model from kaggle
Myhandler.py file

import io
import os
import logging
import torch
import numpy as np
import torch.nn.functional as F
from PIL import Image
from torchvision import transforms,datasets, models
from ts.torch_handler.image_classifier import ImageClassifier
from ts.torch_handler.base_handler import BaseHandler
from ts.utils.util import list_classes_from_module
import importlib
from torch.autograd import Variable
import seaborn as sns
import torchvision
from torch import optim, cuda
from torch.utils.data import DataLoader, sampler
import torch.nn as nn

import warnings
warnings.filterwarnings('ignore', category=FutureWarning)

# Data science tools
import pandas as pd

#path = 'C:\\Users\\fazil\\OneDrive\\Desktop\\pytorch\\vgg11\\vgg16.pt'

path = r'C:\Users\fazil\OneDrive\Desktop\pytorch\vgg11\vgg16.pt'
#image = r'C:\Users\fazil\OneDrive\Desktop\pytorch\vgg11\normal.jpeg'

class VGGImageClassifier(ImageClassifier):
    """
    Overriding the model loading code as a workaround for issue :
    https://github.com/pytorch/serve/issues/535
    https://github.com/pytorch/vision/issues/2473
    """
    def __init__(self):
        self.model = None
        self.mapping = None
        self.device = None
        self.initialized = False

    def initialize(self,context):
        """load eager mode state_dict based model"""
        properties = context.system_properties
        #self.device = torch.device(
        #"cuda:" + str(properties.get("gpu_id"))
        #if torch.cuda.is_available()
        #    else "cpu"
        #)
        model_dir = properties.get("model_dir")

        model_pt_path = os.path.join(model_dir, "model.pt")
        # Read model definition file
        model_def_path = os.path.join(model_dir, "model.py")
        if not os.path.isfile(model_def_path):
            raise RuntimeError("Missing the model definition file")

        checkpoint = torch.load(path, map_location='cpu')
        logging.error('%s ',checkpoint)
        self.model = models.vgg16(pretrained=True)
        logging.error('%s ',self.model)
        self.model.classifier = checkpoint['classifier']
        logging.error('%s ', self.model.classifier )
        self.model.load_state_dict(checkpoint['state_dict'], strict=False)
        self.model.class_to_idx = checkpoint['class_to_idx']
        self.model.idx_to_class = checkpoint['idx_to_class']
        self.model.epochs = checkpoint['epochs']

        optimizer = checkpoint['optimizer']
        optimizer.load_state_dict(checkpoint['optimizer_state_dict'])

        for param in model.parameters():
            param.requires_grad = False

        logger.debug('Model file {0} loaded successfully'.format(model_pt_path))
        self.initialized = True

    def preprocess(self,data):
        image = data.get("data")
        if image is None:
            image = data.get("body")
        image_transform =transforms.Compose([
        transforms.Resize(size=256),
        transforms.CenterCrop(size=224),
        transforms.ToTensor(),
        transforms.Normalize((0.5), (0.5))
        ])
        image = Image.open(io.BytesIO(image)).convert('RGB')
        image = image_transform(image)
        image = image.unsqueeze(0)
        return image

    def inference(self, image):

        outs = self.model.forward(image)
        probs = F.softmax(outs , dim=1)
        preds = torch.argmax(probs, dim=1)
        logging.error('%s ',preds)
        return preds

    def postprocess(self, preds):
        res = []

        preds = preds.cpu().tolist()
        for pred in preds:
            label = self.mapping[str(pred)] [1]
            res.append({'label': label , 'index': pred })
        return res

_service = VGGImageClassifier()

def handle(data,context):
    if not _service.initialized:
        _service.initialize(context)

    if data is None:
        return None
    data = _service.preprocess(data)
    data = _service.inference(data)
    data = _service.postprocess(data)

    return data

this is the error i got
Torchserve version: 0.3.1
TS Home: C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages
Current directory: C:\Users\fazil\OneDrive\Desktop\pytorch\vgg11
Temp directory: C:\Users\fazil\AppData\Local\Temp
Number of GPUs: 0
Number of CPUs: 4
Max heap size: 3038 M
Python executable: c:\users\fazil\anaconda3\envs\serve\python.exe
Config file: ./config.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8081
Metrics address: http://0.0.0.0:8082
Model Store: C:\Users\fazil\OneDrive\Desktop\pytorch\vgg11\model_store
Initial Models: vgg16.mar
Log dir: C:\Users\fazil\OneDrive\Desktop\pytorch\vgg11\logs
Metrics dir: C:\Users\fazil\OneDrive\Desktop\pytorch\vgg11\logs
Netty threads: 32
Netty client threads: 0
Default workers per model: 4
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Prefer direct buffer: false
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: false
Metrics report format: prometheus
Enable metrics API: true
2021-04-08 12:33:22,517 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: vgg16.mar
2021-04-08 12:33:40,392 [INFO ] main org.pytorch.serve.archive.ModelArchive - eTag 85b61fc819804aea9db0ca8786c2e427
2021-04-08 12:33:40,423 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model vgg16
2021-04-08 12:33:40,424 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model vgg16
2021-04-08 12:33:40,424 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model vgg16 loaded.
2021-04-08 12:33:40,426 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: vgg16, count: 4
2021-04-08 12:33:40,481 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: NioServerSocketChannel.
2021-04-08 12:33:41,173 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: None
2021-04-08 12:33:41,177 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: None
2021-04-08 12:33:41,180 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]12328
2021-04-08 12:33:41,180 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]14588
2021-04-08 12:33:41,180 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2021-04-08 12:33:41,181 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2021-04-08 12:33:41,181 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.8
2021-04-08 12:33:41,181 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.8
2021-04-08 12:33:41,186 [DEBUG] W-9001-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - W-9001-vgg16_1.0 State change null -> WORKER_STARTED
2021-04-08 12:33:41,186 [DEBUG] W-9002-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-vgg16_1.0 State change null -> WORKER_STARTED
2021-04-08 12:33:41,199 [INFO ] W-9001-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /127.0.0.1:9001
2021-04-08 12:33:41,199 [INFO ] W-9002-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /127.0.0.1:9002
2021-04-08 12:33:41,240 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: None
2021-04-08 12:33:41,244 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]12008
2021-04-08 12:33:41,244 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2021-04-08 12:33:41,245 [DEBUG] W-9000-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-vgg16_1.0 State change null -> WORKER_STARTED
2021-04-08 12:33:41,245 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.8
2021-04-08 12:33:41,245 [INFO ] W-9000-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /127.0.0.1:9000
2021-04-08 12:33:41,255 [INFO ] W-9003-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: None
2021-04-08 12:33:41,260 [INFO ] W-9003-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]15216
2021-04-08 12:33:41,260 [INFO ] W-9003-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2021-04-08 12:33:41,261 [DEBUG] W-9003-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - W-9003-vgg16_1.0 State change null -> WORKER_STARTED
2021-04-08 12:33:41,261 [INFO ] W-9003-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.8
2021-04-08 12:33:41,262 [INFO ] W-9003-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /127.0.0.1:9003
2021-04-08 12:33:41,768 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:8080
2021-04-08 12:33:41,768 [INFO ] main org.pytorch.serve.ModelServer - Initialize Management server with: NioServerSocketChannel.
2021-04-08 12:33:41,774 [INFO ] main org.pytorch.serve.ModelServer - Management API bind to: http://0.0.0.0:8081
2021-04-08 12:33:41,775 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: NioServerSocketChannel.
2021-04-08 12:33:41,777 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://0.0.0.0:8082
2021-04-08 12:33:41,784 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: ('127.0.0.1', 9001).
2021-04-08 12:33:41,784 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: ('127.0.0.1', 9002).
2021-04-08 12:33:41,784 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: ('127.0.0.1', 9000).
2021-04-08 12:33:41,784 [INFO ] W-9003-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: ('127.0.0.1', 9003).
Model server started.
2021-04-08 12:33:48,486 [INFO ] pool-2-thread-1 TS_METRICS - CPUUtilization.Percent:100.0|#Level:Host|#hostname:fazil,timestamp:1617865428
2021-04-08 12:33:48,487 [INFO ] pool-2-thread-1 TS_METRICS - DiskAvailable.Gigabytes:74.49674987792969|#Level:Host|#hostname:fazil,timestamp:1617865428
2021-04-08 12:33:48,491 [INFO ] pool-2-thread-1 TS_METRICS - DiskUsage.Gigabytes:147.9403419494629|#Level:Host|#hostname:fazil,timestamp:1617865428
2021-04-08 12:33:48,496 [INFO ] pool-2-thread-1 TS_METRICS - DiskUtilization.Percent:66.5|#Level:Host|#hostname:fazil,timestamp:1617865428
2021-04-08 12:33:48,499 [INFO ] pool-2-thread-1 TS_METRICS - MemoryAvailable.Megabytes:4488.515625|#Level:Host|#hostname:fazil,timestamp:1617865428
2021-04-08 12:33:48,504 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUsed.Megabytes:7658.80859375|#Level:Host|#hostname:fazil,timestamp:1617865428
2021-04-08 12:33:48,513 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUtilization.Percent:63.0|#Level:Host|#hostname:fazil,timestamp:1617865428
2021-04-08 12:34:24,385 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Backend worker process died.
2021-04-08 12:34:24,439 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Traceback (most recent call last):
2021-04-08 12:34:24,440 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 182, in <module>
2021-04-08 12:34:24,443 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     worker.run_server()
2021-04-08 12:34:24,444 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 154, in run_server
2021-04-08 12:34:24,446 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.handle_connection(cl_socket)
2021-04-08 12:34:24,446 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 116, in handle_connection
2021-04-08 12:34:24,447 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     service, result, code = self.load_model(msg)
2021-04-08 12:34:24,448 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 89, in load_model
2021-04-08 12:34:24,523 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     service = model_loader.load(model_name, model_dir, handler, gpu, batch_size, envelope)
2021-04-08 12:34:24,582 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "c:\users\fazil\anaconda3\envs\serve\lib\site-packages\ts\model_loader.py", line 104, in load
2021-04-08 12:34:24,597 [INFO ] nioEventLoopGroup-5-2 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2021-04-08 12:34:24,583 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Backend worker process died.
2021-04-08 12:34:24,646 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     initialize_fn(service.context)
2021-04-08 12:34:24,646 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Traceback (most recent call last):
2021-04-08 12:34:24,649 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 182, in <module>
2021-04-08 12:34:24,649 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     worker.run_server()
2021-04-08 12:34:24,650 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 154, in run_server
2021-04-08 12:34:24,648 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "c:\users\fazil\anaconda3\envs\serve\lib\site-packages\ts\model_loader.py", line 131, in <lambda>
2021-04-08 12:34:24,652 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.handle_connection(cl_socket)
2021-04-08 12:34:24,649 [DEBUG] W-9000-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2021-04-08 12:34:24,734 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 116, in handle_connection
2021-04-08 12:34:24,653 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     initialize_fn = lambda ctx: entry_point(None, ctx)
2021-04-08 12:34:24,734 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     service, result, code = self.load_model(msg)
2021-04-08 12:34:24,735 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\AppData\Local\Temp\models\85b61fc819804aea9db0ca8786c2e427\hanndler.py", line 268, in handle
2021-04-08 12:34:24,735 [DEBUG] W-9000-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056)
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133)
        at java.base/java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432)
        at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:188)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
2021-04-08 12:34:24,736 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 89, in load_model
2021-04-08 12:34:24,753 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     service = model_loader.load(model_name, model_dir, handler, gpu, batch_size, envelope)
2021-04-08 12:34:24,736 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     _service.initialize(context)
2021-04-08 12:34:24,754 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\AppData\Local\Temp\models\85b61fc819804aea9db0ca8786c2e427\hanndler.py", line 111, in initialize
2021-04-08 12:34:24,754 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "c:\users\fazil\anaconda3\envs\serve\lib\site-packages\ts\model_loader.py", line 104, in load
2021-04-08 12:34:24,754 [WARN ] W-9000-vgg16_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: vgg16, error: Worker died.
2021-04-08 12:34:24,756 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     initialize_fn(service.context)
2021-04-08 12:34:24,755 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.model.classifier = checkpoint['classifier']
2021-04-08 12:34:24,758 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "c:\users\fazil\anaconda3\envs\serve\lib\site-packages\ts\model_loader.py", line 131, in <lambda>
2021-04-08 12:34:24,810 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     initialize_fn = lambda ctx: entry_point(None, ctx)
2021-04-08 12:34:24,811 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\AppData\Local\Temp\models\85b61fc819804aea9db0ca8786c2e427\hanndler.py", line 268, in handle
2021-04-08 12:34:24,757 [DEBUG] W-9000-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-vgg16_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2021-04-08 12:34:24,871 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     _service.initialize(context)
2021-04-08 12:34:24,872 [WARN ] W-9000-vgg16_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-vgg16_1.0-stderr
2021-04-08 12:34:24,812 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - KeyError: 'classifier'
2021-04-08 12:34:24,872 [WARN ] W-9000-vgg16_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-vgg16_1.0-stdout
2021-04-08 12:34:24,872 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\AppData\Local\Temp\models\85b61fc819804aea9db0ca8786c2e427\hanndler.py", line 111, in initialize
2021-04-08 12:34:24,874 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.model.classifier = checkpoint['classifier']
2021-04-08 12:34:24,903 [INFO ] nioEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9001 Worker disconnected. WORKER_STARTED
2021-04-08 12:34:24,876 [INFO ] W-9000-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 1 seconds.
2021-04-08 12:34:24,931 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - KeyError: 'classifier'
2021-04-08 12:34:24,932 [DEBUG] W-9001-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2021-04-08 12:34:24,974 [DEBUG] W-9001-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056)
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133)
        at java.base/java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432)
        at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:188)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
2021-04-08 12:34:25,015 [WARN ] W-9001-vgg16_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: vgg16, error: Worker died.
2021-04-08 12:34:25,015 [DEBUG] W-9001-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - W-9001-vgg16_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2021-04-08 12:34:25,016 [WARN ] W-9001-vgg16_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9001-vgg16_1.0-stderr
2021-04-08 12:34:25,017 [WARN ] W-9001-vgg16_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9001-vgg16_1.0-stdout
2021-04-08 12:34:25,017 [INFO ] W-9001-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9001 in 1 seconds.
2021-04-08 12:34:25,038 [INFO ] W-9000-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-vgg16_1.0-stdout
2021-04-08 12:34:25,038 [INFO ] W-9000-vgg16_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-vgg16_1.0-stderr
2021-04-08 12:34:25,085 [INFO ] W-9001-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9001-vgg16_1.0-stdout
2021-04-08 12:34:25,085 [INFO ] W-9001-vgg16_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9001-vgg16_1.0-stderr
2021-04-08 12:34:25,247 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Backend worker process died.
2021-04-08 12:34:25,247 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Traceback (most recent call last):
2021-04-08 12:34:25,248 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 182, in <module>
2021-04-08 12:34:25,248 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     worker.run_server()
2021-04-08 12:34:25,248 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 154, in run_server
2021-04-08 12:34:25,249 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.handle_connection(cl_socket)
2021-04-08 12:34:25,250 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 116, in handle_connection
2021-04-08 12:34:25,250 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     service, result, code = self.load_model(msg)
2021-04-08 12:34:25,251 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\anaconda3\envs\serve\Lib\site-packages\ts\model_service_worker.py", line 89, in load_model
2021-04-08 12:34:25,251 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     service = model_loader.load(model_name, model_dir, handler, gpu, batch_size, envelope)
2021-04-08 12:34:25,253 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "c:\users\fazil\anaconda3\envs\serve\lib\site-packages\ts\model_loader.py", line 104, in load
2021-04-08 12:34:25,253 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     initialize_fn(service.context)
2021-04-08 12:34:25,254 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "c:\users\fazil\anaconda3\envs\serve\lib\site-packages\ts\model_loader.py", line 131, in <lambda>
2021-04-08 12:34:25,254 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     initialize_fn = lambda ctx: entry_point(None, ctx)
2021-04-08 12:34:25,254 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\AppData\Local\Temp\models\85b61fc819804aea9db0ca8786c2e427\hanndler.py", line 268, in handle
2021-04-08 12:34:25,255 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     _service.initialize(context)
2021-04-08 12:34:25,256 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "C:\Users\fazil\AppData\Local\Temp\models\85b61fc819804aea9db0ca8786c2e427\hanndler.py", line 111, in initialize
2021-04-08 12:34:25,257 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.model.classifier = checkpoint['classifier']
2021-04-08 12:34:25,257 [INFO ] W-9002-vgg16_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - KeyError: 'classifier'
2021-04-08 12:34:25,454 [INFO ] nioEventLoopGroup-5-4 org.pytorch.serve.wlm.WorkerThread - 9002 Worker disconnected. WORKER_STARTED
2021-04-08 12:34:25,456 [DEBUG] W-9002-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2021-04-08 12:34:25,457 [DEBUG] W-9002-vgg16_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056)
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133)
        at java.base/java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432)
        at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:188)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
2021-04-08 12:34:25,482 [WARN ] W-9002-vgg16_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: vgg16, error: Worker died.

and also i load the model from path
because i got error if i use model_pt_path
can some one help me with this


title: Gradually decay the weight of loss function

body:
I am not sure is the right place to ask this question, feel free to tell me if I need to remove the post.

I am quite new in pyTorch and currently working with CycleGAN (pyTorch implementation) as a part of my project and I understand most of the implementation of cycleGAN.

I read the paper with the name ‘CycleGAN with better Cycles’ and I am trying to apply the modification which mentioned in the paper. One of modification is Cycle consistency weight decay which I don’t know how to apply.

optimizer_G.zero_grad()

# Identity loss
loss_id_A = criterion_identity(G_BA(real_A), real_A)
loss_id_B = criterion_identity(G_AB(real_B), real_B)

loss_identity = (loss_id_A + loss_id_B) / 2

# GAN loss
fake_B = G_AB(real_A)
loss_GAN_AB = criterion_GAN(D_B(fake_B), valid)
fake_A = G_BA(real_B)
loss_GAN_BA = criterion_GAN(D_A(fake_A), valid)

loss_GAN = (loss_GAN_AB + loss_GAN_BA) / 2

# Cycle consistency loss
recov_A = G_BA(fake_B)
loss_cycle_A = criterion_cycle(recov_A, real_A)
recov_B = G_AB(fake_A)
loss_cycle_B = criterion_cycle(recov_B, real_B)

loss_cycle = (loss_cycle_A + loss_cycle_B) / 2

# Total loss
loss_G =    loss_GAN +
            lambda_cyc * loss_cycle + #lambda_cyc is 10
            lambda_id * loss_identity #lambda_id is 0.5 * lambda_cyc

loss_G.backward()
optimizer_G.step()

My question is how can I gradually decay the weight of cycle consistency loss?

Any help in implementing this modification would be appreciated.

This is from the paper:
Cycle consistency loss helps to stabilize training a lot in early stages but becomes an obstacle towards realistic images in later stages. We propose to gradually decay the weight of cycle consistency loss λ as training progress. However, we should still make sure that λ is
not decayed to 0 so that generators won’t become unconstrained and go completely wild.

Thanks in advance.


title: SageMaker PyTorchModel passing custom variables

body:
When deploying a model with SageMaker through the PyTorchModel class, is it possible to pass a custom environmental variable or kwargs?

I'd like to be able to switch the functionality of the serving code via a custom argument rather than needing to write multiple serve.py to handle different training model export methods.

model = PyTorchModel(name='my_model',
                     model_data=estimator.model_data,
                     role=role,
                     framework_version='1.0.0',
                     entry_point='serve.py',
                     source_dir='src',
                     sagemaker_session=sess,
                     predictor_cls=ImagePredictor,
                     <custom_argument?>
                    )


title: modification from classification loss to regression loss

body:
General
I am following this repo for object detection
https://github.com/yhenon/pytorch-retinanet
Motivation
Object detection networks usually perform 2 tasks.For every object in the image output a class confidence score and bounding box regression score.For my task along with these 2 outputs, for every object i want to output one more regression score which will be between 0-5.
Problem statement
The approach i have taken is that since the network already does classifcation i thought i would modify some of the parts and make it a regression loss.The loss used in the repo mentioned above is focal loss.Part of what the classification loss looks like is described below, i would like to modify this to be a regression loss.
     targets = torch.ones(classification.shape) * -1#Here classification shape is = torch.Size([114048, 1])
    targets = targets.cuda()
    targets[torch.lt(IoU_max, 0.4), :] = 0#Iou_max is a tensor of shape = torch.Size([114048]) which looks like tensor([0., 0., 0.,  ..., 0., 0., 0.], device='cuda:0')
    positive_indices = torch.ge(IoU_max, 0.5)
    num_positive_anchors = positive_indices.sum()
    assigned_annotations = bbox_annotation[IoU_argmax, :]#Here bbox_annotation has the shape torch.Size([6, 6]) and IoU_argmax has the shape torch.Size([114048])
    targets[positive_indices, :] = 0#Here positive indices has the shape torch.Size([114048])
    targets[positive_indices, assigned_annotations[positive_indices, 4].long()] = 1
    alpha_factor = torch.ones(targets.shape).cuda() * alpha#here alpha is 0.25
    alpha_factor = torch.where(torch.eq(targets, 1.), alpha_factor, 1. - alpha_factor)
    focal_weight = torch.where(torch.eq(targets, 1.), 1. - classification, classification)
    focal_weight = alpha_factor * torch.pow(focal_weight, gamma)#here gamma is 2.0

    bce = -(targets * torch.log(classification) + (1.0 - targets) * torch.log(1.0 - classification))
    cls_loss = focal_weight * bce
    cls_loss = torch.where(torch.ne(targets, -1.0), cls_loss,torch.zeros(cls_loss.shape).cuda())
  classification_losses.append(cls_loss.sum()/torch.clamp(num_positive_anchors.float(), min=1.0))

So i tried modifying some of the bits described above.But even though the code does not break This does not learn anything.Here erosion is just a tensor but erosion score is something we want to regress.
    targets = torch.ones(erosion.shape) * -1#Here erosion shape is torch.Size([114048, 1])
    targets = targets.cuda()
    targets[torch.lt(IoU_max, 0.4), :] = 0#This part remains the same as above
    positive_indices = torch.ge(IoU_max, 0.5)
    num_positive_anchors = positive_indices.sum()
    assigned_annotations = bbox_annotation[IoU_argmax, :]
    targets[positive_indices, :] = 0
    targets[assigned_annotations[positive_indices, 5].long()] = 1#Here i have indexed along the 5th dimension because the 5th dimension contains annotations for erosion.
    criterion = nn.MSELoss()#This part is where i use the mean squared loss for regression.
    loss = torch.sqrt(criterion(targets, erosion))
    erosion_losses.append(loss)

This will not throw an error , but it does not learn anything either.
Could someone please help me with the correct formulation of this loss to perform regression.
More info
        assigned_annotations_erosion = assigned_annotations[positive_indices, 5]
        erosion_pred =  erosion[positive_indices].view(-1)
        criterion = nn.MSELoss()#
        loss = torch.sqrt(criterion(assigned_annotations_erosion, erosion_pred))
        loss *torch.tensor(5.0).cuda()
        # import pdb;pdb.set_trace()
        erosion_losses.append(loss)


title: RuntimeError when execute torch.matmul()

body:
I’m trying to implement an attention model, but fail to execute the matmul

torch.matmul(att, v)

The shape of att and v is:

att shape:torch.Size([20, 3, 128, 128])
v shape:torch.Size([20, 3, 128, 100])

i get such an error:

  RuntimeError: Expected tensor to have size 100 at dimension 1, but got size 128 for argument #2 ‘batch2’ (while checking arguments for bmm)

I also tried generate two tensors with the same shape by torch.randn and repeat the same operation and no error ouccrred. I don’t know what makes such an error


title: Broadcast error HDF5: Can't broadcast (3, 2048, 1, 1) -> (4, 2048, 1, 1)

body:
I have received the following error:

TypeError: Can't broadcast (3, 2048, 1, 1) -> (4, 2048, 1, 1)

I am extracting features and placing them into a hdf5 dataset like this:

array_40 = hdf5_file.create_dataset(
                    f'{phase}_40x_arrays',  shape, maxshape=(None, args.batch_size, 2048, 1, 1))

In (None, args.batch_size, 2048, 1, 1), None is specified due to the unknown nature of the size of dataset. args.batch_size is 4 in this case, 2048, 1 and 1 are the number of features extracted and their spatial dimensions.

shape is defined as:

shape = (dataset_length, args.batch_size, 2048, 1, 1)

However, I'm not sure what I can do with the args.batch_size, which in this case is 4. I can't leave this as None as it comes up with an illegal error:

ValueError: Illegal value in chunk tuple

EDIT: Yes, you're absolutley right. I'm trying to incrementally write to a hdf5 dataset. I've shown more of the code below. I'm extracting features and storing them incrementally into a hdf5 dataset. Despite a batch size of 4, it would be ideal to save each item from the batch, incrementally as its own instance/row.

shape = (dataset_length, 2048, 1, 1)
            all_shape = (dataset_length, 6144, 1, 1)
            labels_shape = (dataset_length)
            batch_shape = (1,)

            path = args.HDF5_dataset + f'{phase}.hdf5'

            #hdf5_file = h5py.File(path, mode='w')
            with h5py.File(path, mode='a') as hdf5_file:

                array_40 = hdf5_file.create_dataset(
                    f'{phase}_40x_arrays',  shape, maxshape=(None, 2048, 1, 1)
                )
                array_labels = hdf5_file.create_dataset(
                    f'{phase}_labels', labels_shape, maxshape=(None), dtype=string_type
                )
                array_batch_idx = hdf5_file.create_dataset(
                    f'{phase}_batch_idx', data=np.array([-1, ])
                )

                hdf5_file.close()

        # either new or checkpionted file exists
        # load file and create references to exisitng h5 datasets
        with h5py.File(path, mode='r+') as hdf5_file:
            array_40 = hdf5_file[f'{phase}_40x_arrays']
            array_labels = hdf5_file[f'{phase}_labels']
            array_batch_idx = hdf5_file[f'{phase}_batch_idx']

            batch_idx = int(array_batch_idx[0]+1)

            print("Batch ID is restarting from {}".format(batch_idx))

            dataloaders_dict = torch.utils.data.DataLoader(datasets_dict, batch_size=args.batch_size, sampler=SequentialSampler2(
                datasets_dict, batch_idx, args.batch_size),drop_last=True, num_workers=args.num_workers, shuffle=False)  # make sure shuffling is false for sampler to work and incase you restart

            for i, (inputs40x, paths40x, labels) in enumerate(dataloaders_dict):

                print(f'Batch ID: {batch_idx}')

                inputs40x = inputs40x.to(device)
                labels = labels.to(device)
                paths = paths40x

                x40 = resnet(inputs40x)

                # torch.Size([1, 2048, 1, 1]) batch, feats, 1l, 1l
                array_40[...] = x40.cpu()
                array_labels[batch_idx, ...] = labels[:].cpu()
                array_batch_idx[:,...] = batch_idx

                batch_idx +=1
                hdf5_file.flush()


title: how to load mobilenet with pytorch

body:
I try to do the following:

import torchvision.models as models
mobilenet = models.mobilenet_v2(pretrained=True)

And then i get the error:

AttributeError: module 'torchvision.models' has no attribute 'mobilenet_v2'

What can I do to fix this?


title: Getting Unknown type annotation error when JIT saving pytorch model

body:
When JIT saving “model.pt” of a complex pytorch model with many custom classes, I am encountering the error that pytorch doesn’t know the type annotation of one of those custom classes. In other words, the following code (drastically summarized from original) fails on the seventh line:
import torch
from gan import Generator
from gan.blocks import SpadeBlock

generator = Generator()
generator.load_weights("path/to/weigts")
jitted = torch.jit.script(generator)
torch.jit.save(jitted, "model.pt")

Error:
Traceback (most recent call last):
  File "pth2onnx.py", line 72, in <module>
    to_torch_jit(generator)
  File "pth2onnx.py", line 24, in to_torch_jit
    jitted = torch.jit.script(generator)
  File "/home/a.nieuwland/.conda/envs/python3.6/lib/python3.6/site-packages/torch/jit/__init__.py", line 1516, in script
    return torch.jit._recursive.create_script_module(obj, torch.jit._recursive.infer_methods_to_compile)
  File "/home/a.nieuwland/.conda/envs/python3.6/lib/python3.6/site-packages/torch/jit/_recursive.py", line 310, in create_script_module
    concrete_type = concrete_type_store.get_or_create_concrete_type(nn_module)
  File "/home/a.nieuwland/.conda/envs/python3.6/lib/python3.6/site-packages/torch/jit/_recursive.py", line 269, in get_or_create_concrete_type
    concrete_type_builder = infer_concrete_type_builder(nn_module)
  File "/home/a.nieuwland/.conda/envs/python3.6/lib/python3.6/site-packages/torch/jit/_recursive.py", line 138, in infer_concrete_type_builder
    sub_concrete_type = concrete_type_store.get_or_create_concrete_type(item)
  File "/home/a.nieuwland/.conda/envs/python3.6/lib/python3.6/site-packages/torch/jit/_recursive.py", line 269, in get_or_create_concrete_type
    concrete_type_builder = infer_concrete_type_builder(nn_module)
  File "/home/a.nieuwland/.conda/envs/python3.6/lib/python3.6/site-packages/torch/jit/_recursive.py", line 138, in infer_concrete_type_builder
    sub_concrete_type = concrete_type_store.get_or_create_concrete_type(item)
  File "/home/a.nieuwland/.conda/envs/python3.6/lib/python3.6/site-packages/torch/jit/_recursive.py", line 269, in get_or_create_concrete_type
    concrete_type_builder = infer_concrete_type_builder(nn_module)
  File "/home/a.nieuwland/.conda/envs/python3.6/lib/python3.6/site-packages/torch/jit/_recursive.py", line 126, in infer_concrete_type_builder
    attr_type = infer_type(name, item)
  File "/home/a.nieuwland/.conda/envs/python3.6/lib/python3.6/site-packages/torch/jit/_recursive.py", line 99, in infer_type
    attr_type = torch.jit.annotations.ann_to_type(class_annotations[name], _jit_internal.fake_range())
  File "/home/a.nieuwland/.conda/envs/python3.6/lib/python3.6/site-packages/torch/jit/annotations.py", line 303, in ann_to_type
    raise ValueError("Unknown type annotation: '{}'".format(ann))
ValueError: Unknown type annotation: '<class 'gan.blocks.SpadeBlock'>'

The type it complains about is indeed a class we ourselves have programmed and used in the loaded Generator. I would appreciate pointers on what could cause this or how to investigate this!
I tried the following:

explicitly importing SpadeBlock in the script that calls torch.jit.script
ensured it inherits from nn.Module (as does Generator)
ensured the gan package is installed, using pip install --user -e

Any ideas? Thanks in advance!


title: Pytorch NLP sequence length of target in Transformer

body:
I'm trying to understand the code of Transformer (https://github.com/SamLynnEvans/Transformer).
If seeing the train_model function in "train" script, I wonder why need to use the different sequence length of trg_input from trg:
trg_input = trg[:, :-1]

In this case, the sequence length of trg_input is "seq_len(trg) - 1".
It means that trg is like:
<sos> tok1 tok2 tokn <eos>

and trg_input is like:
<sos> tok1 tok2 tokn    (no eos token)

Please let me know the reason.
Thank you.
The related code is like below:
    for i, batch in enumerate(opt.train):
        src = batch.src.transpose(0, 1).to('cuda')
        trg = batch.trg.transpose(0, 1).to('cuda')

        trg_input = trg[:, :-1]
        src_mask, trg_mask = create_masks(src, trg_input, opt)
        preds = model(src, trg_input, src_mask, trg_mask)
        ys = trg[:, 1:].contiguous().view(-1)
        opt.optimizer.zero_grad()
        loss = F.cross_entropy(preds.view(-1, preds.size(-1)), ys, ignore_index=opt.trg_pad)
        loss.backward()
        opt.optimizer.step()

def create_masks(src, trg, opt):

    src_mask = (src != opt.src_pad).unsqueeze(-2)

    if trg is not None:
        trg_mask = (trg != opt.trg_pad).unsqueeze(-2)
        size = trg.size(1) # get seq_len for matrix
        np_mask = nopeak_mask(size, opt)
        if trg.is_cuda:
            np_mask.cuda()
        trg_mask = trg_mask & np_mask

    else:
        trg_mask = None
    return src_mask, trg_mask


title: how to fix the torch.cuda.is_available() False problem without restarting the machine?

body:
I have:
$ python
Python 3.7.6 (default, Jan  8 2020, 19:59:22)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
False
>>> quit()

$ nvidia-smi
Wed Oct 14 21:28:50 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.51.06    Driver Version: 450.51.06    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 2070    Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   47C    P8     9W /  N/A |   1257MiB /  7982MiB |     11%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1424      G   /usr/lib/xorg/Xorg                823MiB |
|    0   N/A  N/A      1767      G   /usr/bin/gnome-shell              407MiB |
|    0   N/A  N/A      6420      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      6949      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      7447      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      8888      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      9218      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      9282      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A     65854      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A     70801      G   /usr/lib/firefox/firefox            2MiB |
+-----------------------------------------------------------------------------+

Is there a way I could fix this problem without having to restart my machine?
I have Ubuntu 20.04 and PyTorch 1.6.0
After I restarted the machine, here's what I get:


title: Pytorch Lightning repeats main script for each gpu

body:
I have a single machine with two graphic cards. I'd like to use both for training my model. In an attempt to make my life easier, I'm using pytorch lightning.
When I run my script without gpus, everything works fine:
trainer = Trainer(gpus=None)

But I'd like to run it on multiple gpus. Confusingly, the manual, first states that for one machine with multiple gpus you should use the 'dp' accelerator:
trainer = Trainer(gpus=2, accelerator='dp')

This starts the training session, but gives me an BroadcastBackward error that I only get when I try to use the dp accelerator (details below).
However the manual goes on to say that

DP use is discouraged by PyTorch and Lightning. Use DDP which is more
stable and at least 3x faster

Sounds great! So I do:
trainer = Trainer(gpus=2, accelerator='ddp')

But now when I run trainer.fit, the trainer starts a second process (proc = subprocess.Popen(command, env=env_copy, cwd=cwd) in ddp_accelerator.py.
On the one hand this is expected behaviour (one process per gpu), but on the other hand, it really literary reruns my main script (it's not just re-imported, so protecting with if __name__ == "__main__": doesn't help). My script is an optimisation scheme, with the following flow:

generate hyperparameters (based on results of earlier runs)
build, train, evaluate model: return performance measure
repeat from 1

This means that the second process gets different hyperparameters than the first, and this obviously throws an error.
Running on 1 gpu works fine by the way (training completes without errors):
trainer = Trainer(gpus=1, accelerator='ddp')

How should I approach this problem? Stick with 'dp' and fix my BroadcastBackward error? Or stick with ddp, but how can I then fix the script duplication error?
dp error details:

In file X, Line X, in forward self.RMr.lerp_(Mr.squeeze(), exponential_average_factor) RuntimeError:
Output 0 of BroadcastBackward is a view and is being modified inplace.
This view is the output of a function that returns multiple views.
Such functions do not allow the output views to be modified inplace.
You should replace the inplace operation by an out-of-place one.

system details:
ubuntu, python: 3.8, pytorch-lightning: 1.0.7, pytorch: 1.6.0


title: Is Seq2Seq the right model for my data?

body:
I'm trying to train a model to predict design patterns from web pages. I'm using coordinates of bounding rects given a bunch of element groupings. Patterns look like this:

 [[elementId, width, height, x, y]]

so my target would be the [[x,y]] given [[elementId, width, height]].

Concretely:

 [[5, 1.0, 1.0], [4, 1.0, 1.0], [2, 175.0, 65.0], [2, 1.0, 1.0], [4, 1.0, 1.0]]
 ->
 [[0.0, 0.0], [0.0, 10.0], [3.0, 0.0], [0.0, 68.0], [0.0, 10.0]]

 [[2, 14.0, 14.0], [2, 14.0, 14.0], [2, 14.0, 14.0]]
 ->
 [[0.0, 3.0], [0.0, 3.0], [0.0, 3.0]]

Patterns vary in size so I've padded them with [[0,0,0]]. I currently have about 15k of them, but can get more.

I was told that seq2seq with attention is the right model for this job. I've started with https://machinelearningmastery.com/develop-encoder-decoder-model-sequence-sequence-prediction-keras/ and achieved horrendous results.

Every seq2seq example i can find (searching for keras or pytorch) is used for translation which is categorical and I'm struggling to find a good regression based example.

So my question is:

1) Is this the right model (encoder/decoder LSTM) for what i'm trying to do?

2) Is there any examples if so?

Thanks so much in advance. I don't expect anyone to solve my problem, but any help would be much appreciated!


title: How to (quickly) extract bilinear-interpolated patches from a 2d image at specific points?

body:
Update: The original question formulation was a bit unclear. I am not just cropping the image but applying bilinear interpolation during the patches extraction process. (See the paper reference below). That's why the algorithm is a bit more involved than just taking slices.

I am trying to train a deep learning model to predict face landmarks following this paper. I need to crop parts of the image that contains face into smaller patches around facial landmarks. For example, if we have the image shown below:

The function should generate N=15 "patches", one patch per landmark:

I have the following naïve implementation build on top of torch tensors:

def generate_patch(x, y, w, h, image):
    c = image.size(0)
    patch = torch.zeros((c, h, w), dtype=image.dtype)
    for q in range(h):
        for p in range(w):
            yq = y + q - (h - 1)/2
            xp = x + p - (w - 1)/2
            xd = 1 - (xp - math.floor(xp))
            xu = 1 - (math.ceil(xp) - xp)
            yd = 1 - (yq - math.floor(yq))
            yu = 1 - (math.ceil(yq) - yq)
            for idx in range(c):
                patch[idx, q, p] = (
                    image[idx, math.floor(yq), math.floor(xp)]*yd*xd +
                    image[idx, math.floor(yq),  math.ceil(xp)]*yd*xu +
                    image[idx,  math.ceil(yq), math.floor(xp)]*yu*xd +
                    image[idx,  math.ceil(yq),  math.ceil(xp)]*yu*xu
                ).item()
    return patch

def generate_patches(image, points, n=None, sz=31):
    if n is None:
        n = len(points)//2
    patches = []
    for i in range(n):
        x_val, y_val = points[i], points[i + n]
        patch = generate_patch(x_val, y_val, sz, sz, image)
        patches.append(patch)
    return patches

The code does its work but too slowly. I guess because of all these for-loops and separate pixels indexing. I would like to vectorize this code, or maybe find some C-based implementation that could do it faster.

I know there is the extract_patches_2d function from sklearn package that helps to pick random patches from the image. However, I would like to pick the patches from specific points instead of doing it randomly. I guess that I can somehow adapt the aforementioned function, or convert the implementation shown above into Cython/C code but probably someone has already done something like this before.

Could you please advise some alternative to the code shown above, or maybe a proposal on how to make it faster? (Except using several parallel workers).


title: Combination of torch.cat and torchvision.transforms leeds to zero tensor

body:
I want to add some more information to an image in a fourth layer of a tensor which first three layers are based on an image. Afterwards I want to cut a peace out of an image (data augmentation) and have to resize the image to a given size.

For this I created a tensor from a picture and joined it with a tensor with one layer of additional information using torch.cat. (Almost but not all entries of the second tensor have been zeros.)

I sent the result through a transforms.compose (to cut and resize the tensor) But after that the tensor completely consisted out of zeros.

Here I have built a reproducible example.

import torch
from torchvision import transforms

height = 2
width = 4
resize = 2
tensor3 = torch.rand(3,height,width)
tensor1 = torch.zeros(1,height,width)
#tensor1 = torch.rand(1,height,width)

imageToTensor = transforms.ToTensor()
tensorToImage = transforms.ToPILImage()

train_transform = transforms.Compose([
    transforms.RandomResizedCrop(resize, scale=(0.9, 1.0)),
    transforms.ToTensor(),
])

tensor4 = torch.cat((tensor3,tensor1),0)

image4 = tensorToImage(tensor4)
transformed_image4 = train_transform(image4)

print(tensor4)
print(transformed_image4)

tensor([[[0.6774, 0.5293, 0.4420, 0.2463],
         [0.1391, 0.7481, 0.3436, 0.9391]],

        [[0.0652, 0.2061, 0.2931, 0.6126],
         [0.2618, 0.3506, 0.5095, 0.7351]],

        [[0.8555, 0.6320, 0.9461, 0.0928],
         [0.2094, 0.3944, 0.0528, 0.7900]],

        [[0.0000, 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000, 0.0000]]])

tensor([[[0., 0.],
         [0., 0.]],

        [[0., 0.],
         [0., 0.]],

        [[0., 0.],
         [0., 0.]],

        [[0., 0.],
         [0., 0.]]])

If I choose “tensor1 = torch.rand(1,height,width)” I do not have this problem. But if most entries are zero I have.
With scale=(0.5, 1.0) I don't have the problem either.

No some questions:

How may I get the first three layers just resized with non zero entries?
Did I misunderstand something, or is it really weird?


title: How to accelerate the code below? Implementing maxpool without center element

body:
I know maxpool and I am using it in pytorch. Maxpool with dilated parameter is as belows:

Now I want a special form of maxpool, doing maxpool with out the central element. That is the kernel size is 3X3 but the central element should be deleted. Thus the result should come from the rest 8 elements.
Now I am using a for loop, how to accelerate this using numpy or pytorch or anything else?

import numpy as np
from timeit import default_timer as timer

def MaxPool_special(kh, kw, arr):
    """
    to do maxpool without central element
    :param kh:  should always be 3
    :param kw:  should always be 3
    :param arr:  the input array
    :return: arr_res: output array
    """
    h, w = arr.shape[:2]

    arr_res = np.array([[maxpool_ij(i, j, arr, kh, kw) for j in range(w)] for i in range(h)])

    return arr_res

def maxpool_ij(i, j, arr, dh, dw):
    """
    find the maximum value around point(i,j) with dilated parameter
    """
    Mmax = None
    imin, imax = i - dh, i + dh
    jmin, jmax = j - dw, j + dw
    if imin >= 0 and imax < h and jmin >= 0 and jmax < w:
        Mmax = np.max(
            arr[[imin, imin, imin, i, i, imax, imax, imax], [jmin, j, jmax, jmin, jmax, jmin, j, jmax]])
    elif imin < 0 and jmin < 0:
        Mmax = np.max(arr[[i, imax, imax], [jmax, j, jmax]])
    elif imin < 0 and jmax >= w:
        Mmax = np.max(arr[[i, imax, imax], [jmin, jmin, j]])
    elif imax >= h and jmin < 0:
        Mmax = np.max(arr[[imin, imin, i], [j, jmax, jmax]])
    elif imax >= h and jmax >= w:
        Mmax = np.max(arr[[imin, imin, i], [jmin, j, jmin]])
    elif imin < 0:
        Mmax = np.max(arr[[i, i, imax, imax, imax], [jmin, jmax, jmin, j, jmax]])
    elif imax >= h:
        Mmax = np.max(arr[[imin, imin, imin, i, i], [jmin, j, jmax, jmin, jmax]])
    elif jmin < 0:
        Mmax = np.max(arr[[imin, imin, i, imax, imax], [j, jmax, jmax, j, jmax]])
    elif jmax >= w:
        Mmax = np.max(arr[[imin, imin, i, imax, imax], [jmin, j, jmin, jmin, j]])

    assert Mmax, f'Wrong logic above!{imin, imax, jmin, jmax, h, w}'

    return Mmax

#  generate input array
h, w = 400, 500
arr = np.random.randint(0, 256, h * w).reshape(h, w)

tic = timer()
grayPool = MaxPool_special(3, 3, arr)
toc = timer()
print(f'time cost for for-loops: {toc - tic}')

Please help me accelerating this code, thanks!


title: What is the correct way to measure the total execution time for a pytorch function running on GPU?

body:
Following is an example code showing what I am trying to measure. Here I am using time.perf_counter() to measure time. Is this the correct way to measure execution time in this scenario? If not, what is the correct way? My concern is, GPU evaluations are asynchronous and GPU execution might not be completed when ExecTime is measured below.

import torch
import torch.nn.functional as F
import time

Device = torch.device("cuda:0")
ProblemSize = 100
NumChannels = 5
NumFilters = 96
ClassType = torch.float32

X = torch.rand(1, NumChannels, ProblemSize, ProblemSize, dtype=ClassType).to(Device)
weights = torch.rand(NumFilters, NumChannels, 10, 10, dtype=ClassType).to(Device)

#warm up
Y = F.conv2d(X, weights)
Y = F.conv2d(X, weights)

#time
t = time.perf_counter()
Y = F.conv2d(X, weights)
ExecTime = time.perf_counter() - t


title: element wise multiplication of 2D tensors as layer of neural network in pytorch

body:
I have a 3D torch tensor with dimension of [Batch_size, n, n] wich is the out put of a layer of my network and a constant 2D torch tensor with size of [n, n]. How can I perform element wise multiplication over the batch size which should resulted in a torch tensor with size of [Batch_size, n, n]?
I know it is possible to implement this operation using explicit loop but I am interested in most efficient way.

Thanks


title: Vectorized way to shuffle a given tensor using pytorch

body:
I have a tensor A of shape (1,12,2,2) as follows:

  ([[[[1., 3.],
      [9., 11.],

     [[ 2.,  4.],
      [10., 12.]],

     [[ 5.,  7.],
      [13., 15.]],

     [[ 6.,  8.],
      [14., 16.]],

     [[17., 19.],
      [25., 27.]],

     [[18., 20.],
      [26., 28.]],

     [[21., 23.],
      [29., 31.]],

     [[22., 24.],
      [30., 32.]],

     [[33., 35.],
      [41., 43.]],

     [[34., 36.],
      [42., 44.]],

     [[37., 39.],
      [45., 47.]],

     [[38., 40.],
      [46., 48.]]]])

I want to shuffle it using pytorch to produce the following tensor B of shape (1,3,4,4):

tensor([[[[ 1.,  6.,  3.,  8.],
          [21., 34., 23., 36.],
          [ 9., 14., 11., 16.],
          [29., 42., 31., 44.]],

         [[ 2., 17.,  4., 19.],
          [22., 37., 24., 39.],
          [10., 25., 12., 27.],
          [30., 45., 32., 47.]],

         [[ 5., 18.,  7., 20.],
          [33., 38., 35., 40.],
          [13., 26., 15., 28.],
          [41., 46., 43., 48.]]]])

I have implemented this using two for loops as follows:

B = torch.zeros(1,3,4,4, dtype=torch.float)
ctr = 0
for i in range(2):
    for j in range(2):
        B[:,:,i:4:2,j:4:2] = A[:,ctr:ctr+3,:,:]
        ctr = ctr+3

I'm looking for any way to implement this in a vectorized manner in pytorch without these for loops. Maybe using functions like .permute(), etc.


title: Clearing CUDA memory permanently

body:
This is similar to How to clear Cuda memory in PyTorch.

I keep getting the CUDA out of memory error, even though I have used torch.cuda.empty_cache() as the first line of my code, after all the import commands. Also, this error is quite random, and I see a lot of people facing this error on other forums. Isn't there a permanent solution to this?

I also do with torch.no_grad() during evaluation. But this is irrelevant, since my code should begin training at least. Which it doesn't. I have obviously restarted the kernel multiple times and cleared out the cache as mentioned before. Any stored tensors in GPU memory should ideally be cleared by this process.

The error:

RuntimeError: CUDA out of memory. Tried to allocate 144.00 MiB (GPU 0; 11.00 GiB total capacity; 3.99 GiB already allocated; 93.40 MiB free; 4.04 GiB reserved in total by PyTorch)

After multiple attempts of restarting the kernel and clearing out cache, I inserted print(torch.cuda.is_available()) before training begins. It magically worked! Although I do not see any logic behind it.


title: Pytorch: TypeError: copy_(): argument 'other' (position 1) must be Tensor, not Vectors

body:
I am building my model in google colab.

I have created a custom embedding matrix

import torchtext.vocab as vocab

custom_embeddings = vocab.Vectors(name = 'custom_embeddings.txt')
TEXT.build_vocab(train_data, vectors = custom_embeddings)

Here is the code for the Encoder class:

class Encoder(nn.Module):
    def __init__(self,
                 input_dim,
                 hid_dim,
                 n_layers,
                 n_heads,
                 pf_dim,
                 dropout,
                 device,
                 max_length = 100):
        super().__init__()

        self.device = device

        self.tok_embedding = nn.Embedding(input_dim, hid_dim)

        # step added for custom embedding
        self.tok_embedding.weight.data.copy_(custom_embeddings)

        self.pos_embedding = nn.Embedding(max_length, hid_dim)

        self.layers = nn.ModuleList([EncoderLayer(hid_dim,
                                                  n_heads,
                                                  pf_dim,
                                                  dropout,
                                                  device)
                                     for _ in range(n_layers)])

        self.dropout = nn.Dropout(dropout)

        self.scale = torch.sqrt(torch.FloatTensor([hid_dim])).to(device)

    def forward(self, src, src_mask):

        #src = [batch size, src len]
        #src_mask = [batch size, src len]

        batch_size = src.shape[0]
        src_len = src.shape[1]

        pos = torch.arange(0, src_len).unsqueeze(0).repeat(batch_size, 1).to(self.device)

        #pos = [batch size, src len]

        src = self.dropout((self.tok_embedding(src) * self.scale) + self.pos_embedding(pos))

        #src = [batch size, src len, hid dim]

        for layer in self.layers:
            src = layer(src, src_mask)

        #src = [batch size, src len, hid dim]

        return src

Now when I am trying to create the Encoder object, I am getting error for the custom embedding I have used.

enc = Encoder(INPUT_DIM,
              HID_DIM,
              ENC_LAYERS,
              ENC_HEADS,
              ENC_PF_DIM,
              ENC_DROPOUT,
              device)

Error decscription:

TypeError                                 Traceback (most recent call last)
<ipython-input-72-06d3631c029b> in <module>()
     18               ENC_PF_DIM,
     19               ENC_DROPOUT,
---> 20               device)
     21
     22 dec = Decoder(OUTPUT_DIM,

<ipython-input-59-6c2f23451d01> in __init__(self, input_dim, hid_dim, n_layers, n_heads, pf_dim, dropout, device, max_length)
     16
     17         # step added for custom embedding
---> 18         self.tok_embedding.weight.data.copy_(custom_embeddings)
     19
     20         self.pos_embedding = nn.Embedding(max_length, hid_dim)

TypeError: copy_(): argument 'other' (position 1) must be Tensor, not Vectors

Could you please help me to fix this error?

Thanks in advance!


title: ValueError: axes don't match array Pytorch Transpose operation

body:
I'm trying to convert a PIL image to a torch Variable type. Here is the code for it:-

def preprocess_image(pil_im, resize_im=True):
    """
        Processes image for CNNs
    Args:
        PIL_img (PIL_img): PIL Image or numpy array to process
        resize_im (bool): Resize to 224 or not
    returns:
        im_as_var (torch variable): Variable that contains processed float tensor
    """
    # mean and std list for channels (Imagenet)
    mean = [0.485, 0.456, 0.406]
    std = [0.229, 0.224, 0.225]

    #ensure or transform incoming image to PIL image
    if type(pil_im) != Image.Image:
        try:
            pil_im = Image.fromarray(pil_im)
        except Exception as e:
            print("could not transform PIL_img to a PIL Image object. Please check input.")

    # Resize image
    if resize_im:
        pil_im = pil_im.resize((224, 224), Image.ANTIALIAS)

    im_as_arr = np.float32(pil_im)
    print(im_as_arr.shape)
    im_as_arr = im_as_arr.transpose(2, 0, 1)  # Convert array to D,W,H
    # Normalize the channels
    for channel, _ in enumerate(im_as_arr):
        im_as_arr[channel] /= 255
        im_as_arr[channel] -= mean[channel]
        im_as_arr[channel] /= std[channel]
    # Convert to float tensor
    im_as_ten = torch.from_numpy(im_as_arr).float()
    # Add one more channel to the beginning. Tensor shape = 1,3,224,224
    im_as_ten.unsqueeze_(0)
    # Convert to Pytorch variable
    im_as_var = Variable(im_as_ten, requires_grad=True)
    return im_as_var

original_image =  Image.open('blahblah.jpeg')
prep_img = preprocess_image(original_image)

I'm getting an error which says

ValueError                                Traceback (most recent call last)
<ipython-input-27-08cf62156870> in <module>()
      1 original_image =  Image.open('blahblah.jpeg')
----> 2 prep_img = preprocess_image(original_image)

<ipython-input-22-ad146391ce9d> in preprocess_image(pil_im, resize_im)
    155     im_as_arr = np.float32(pil_im)
    156     print(im_as_arr.shape)
--> 157     im_as_arr = im_as_arr.transpose(2, 0, 1)  # Convert array to D,W,H
    158     # Normalize the channels
    159     for channel, _ in enumerate(im_as_arr):

ValueError: axes don't match array

I think there's some issue with the transpose operation. But I tested it out on another use-case and this worked perfectly. Not sure what's triggering this error.


title: How to efficiently calculate pairwise intersection of nonzero indices in a scipy.csr sparse matrix?

body:
I have a scipy.sparse.csr matrix X which is n x p. For each row in X I would like to compute the intersection of the non zero element indices with each row in X and store them in a new tensor or maybe even a dictionary. For example, X is:

X = [
[0., 1.5, 4.7],
[4., 0., 0.],
[0., 0., 2.6]
]

I would like the output to be

intersect =
[
[[1,2], [], [2]],
[[], [0], []],
[[2], [], [2]]
]

intersect[i,j] is an ndarray representing the intersection of the indices of nonzero elements of ith and jth rows of X i.e X[i], X[j].

Currently the way I am doing this is by looping and I would like to vectorize this as it would be much faster and the computations are done in parallel.

# current code
n = X.shape[0]
intersection_dict = {}
for i in range(n):
    for j in range(n):
        indices = np.intersect1d(X[i].indices, X[j].indices)
        intersection_dict[(i,j)] = indices

My n is pretty large so looping over n^2 is very poor. I am just having trouble figuring out a way to vectorize this operation. Does anybody have any ideas on how to tackle this?

EDIT:
It was made apparent that I should explain the problem I am trying to solve, so here it is.

I am solving an optimization problem and have an equation
W = X diag(theta) X'. I want to find W in a quick manner as I update the entries of theta till convergence. Further I am updating parameters using pytorch where sparse operations are not as extensive as in scipy.

where:

X : n x p sparse data matrix (n documents, p features)
theta : p x 1 parameter vector I want to learn and will be updating
X' : p x n transpose of sparse data matrix

note p >> n

I had in mind two methods of solving this quickly

Cache sparse outer product of (see More efficient matrix multiplication with diagonal matrix)
W_ij = X_i * theta * X_j (element wise product of row i of X, theta, and row j of X. And since X_i, X_j are sparse I was thinking if I take the intersection of the nonzero indices then I can do a simple dense elementwise product (sparse element wise product not supported in pytorch) of X_i[intersection indices] * theta[intersection indices] X_j[intersection indices]

I want to vectorize as much of this computation as possible rather than loop as my n is typically in the thousands and p is 11 million.

I am attempting method 2 over method 1 do to the lack of sparse support in Pytorch. Mainly when updating the entries of theta I would not like to do sparse-dense or sparse-sparse operations. I want to do dense-dense operations.


title: tensorboard --logdir=runs not working: Abort trap: 6

body:
I am trying to run tensorboard: tensorboard --logdir=runs.
I have also tried: tensorboard --logdir=runs --host=127.0.0.1.
I am running the command from the terminal from within the the directory, which contains the runs folder.

I get the following error:

[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/descriptor.cc:1367]
CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException:
CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
Abort trap: 6

My Python code contains the following lines:

tb_path = './runs/SimpleLSTM_MNIST'
if os.path.isdir(tb_path):
    shutil.rmtree(tb_path)

writer = tb.SummaryWriter(log_dir=tb_path)

My runs folder contains the folder SimpleLSTM_MNIST, which contains events.out.tfevents.1591953948.computername.local.29440.0.

Operating System: MacOS Catalina

How can I resolve the problem?


title: How to train on single image depth estimation on KITTI dataset with masking method

body:
I'm studying on a deep learning(supervised-learning) to estimate depth images from monocular images.
And the dataset currently uses KITTI data. RGB images (input image) are used KITTI Raw data, and data from the following link is used for ground-truth.
In the process of learning a model by designing a simple encoder-decoder network, the result is not so good, so various attempts are being made.
While searching for various methods, I found that groundtruth only learns valid areas by masking because there are many invalid areas, i.e., values that cannot be used, as shown in the image below.

So, I learned through masking, but I am curious about why this result keeps coming out.

and this is my training part of code.
How can i fix this problem.
for epoch in range(num_epoch):
model.train() ### train ###
for batch_idx, samples in enumerate(tqdm(train_loader)):
    x_train = samples['RGB'].to(device)
    y_train = samples['groundtruth'].to(device)

    pred_depth = model.forward(x_train)
    valid_mask = y_train != 0     #### Here is masking

    valid_gt_depth = y_train[valid_mask]
    valid_pred_depth = pred_depth[valid_mask]

    loss = loss_RMSE(valid_pred_depth, valid_gt_depth)


title: Pytorch reshape to modify batch size in forward()

body: I have a tensor of shape, say: [4,10] where 4 is the batch size and 10 is the length of my input samples buffer. Now, I know that it is really [4,5+5] i.e. the input samples buffer consists of two windows of length 5 which can be processed independently and, best, in parallel. What I am doing is, inside forward() of my model I first reshape the tensor to [8,5], run my layers on it, and then reshape it back to [4,-1] and return. What I am hoping to get from this is Pytorch would run my model on each of the windows (kind of sub-batches) in parallel, effectively yielding a parallel-for loop. It runs OK, Pytorch does not complain or anything but I am getting weird results. I'd like to know if Pytorch can work this way before I dive into debugging my model.


title: Minimization and maximization at the same time in PyTorch

body:
I would like to know how to take gradient steps for the following mathematical operation in PyTorch (A, B and C are PyTorch modules whose parameters do not overlap)

This is somewhat different than the cost function of a Generative Adversarial Network (GAN), so I cannot use examples for GANs off the shelf, and I got stuck while trying to adapt them for the above cost.

One approach I thought of is to construct two optimizers. Optimizer opt1 has the parameters for the modules A and B, and optimizer opt2 has the parameters of module C. One can then:

take a step for minimizing the cost function for C
run the network again with the same input to get the costs (and intermediate outputs) again
take a step with respect to A and B.

I am sure they must be a better way to do this with PyTorch (maybe using some detach operations), possibly without running the network again. Any help is appreciated.


title: what should i add in model initialization in torch.nn?

body:
what should be added in model initialization?  do the LeakyReLU and Dropout have learnable  parameters in
the given model like weights of the Linear layer ?

class Discriminator(nn.Module):
    def __init__(self):
        super(Discriminator, self).__init__()

        self.label_embedding = nn.Embedding(opt.n_classes, opt.n_classes)
        self.lin= nn.LeakyReLU(0.2, inplace=True)
        print( self.lin.parameters)
        self.model = nn.Sequential(
            nn.Linear(opt.n_classes + int(np.prod(img_shape)), 512),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Linear(512, 512),
            nn.Dropout(0.4),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Linear(512, 512),
            nn.Dropout(0.4),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Linear(512, 1),
        )


title: Multi-task learning with mutually exclusive tasks?

body: I wish to perform multi-task learning by performing segmentation and classification using shared layers as shown in the figure. However, the output 1 will be a segmentation mask that works with a different loss function like Dice loss and output 2 works with softmax activations to deliver class probabilities. Also, the data are mutually exclusive, the segmentation task comes from natural image domain, and classification from medical image domain. Is Keras or Pytorch have a defined way or examples demonstrating such multi-task learning tasks?


title: Alternatively train multi task learning model in pytorch - weight updating

body:
I want to build a multi task learning model on two related datasets with different inputs and targets. The two tasks are sharing lower-level layers but with different header layers, a minimal example:
class MultiMLP(nn.Module):
    """
    A simple dense network for MTL on hard parameter sharing.
    """
    def __init__(self):
        super().__init__()
        self.hidden = nn.Linear(100, 200)
        self.out_task0= nn.Linear(200, 1)
        self.out_task0= nn.Linear(200, 1)

    def forward(self, x):
        x = self.hidden(x)
        x = F.relu(x)
        y_task0 = self.out_task0(x)
        y_task1 = self.out_task1(x)
        return [y_task0, y_task1]

The dataloader is constructed so that the batches are alternatively generated from two datasets, i.e. batch 0, 2, 4, ... from task 0, batch 1, 3, 5, ... from task 1.  I wanted to train the network in this way: only update weights for hidden layer and out_task0 for batches from task 0, and update only hidden and out_task1 for task 1.
I then alternatively switch requires_grad for the corresponding tasks during training as following. But I observed that all weights are updated for every iteration.
...
criterion = MSELoss()
for i, data in enumerate(combined_loader):
    x, y = data[0], data[1]
    optimizer.zero_grad()
    # controller is 0 for task0, 1 for task1
    # altenate the header layer
    controller = i % 2
    task0_mode = True if controller == 0 else False
    for name, param in model.named_parameters():
        if name in ['out_task0.weight', 'out_task0.bias']:
            param.requires_grad = task0_mode
        elif name in ['out_task1.weight', 'out_task1.bias']:
            param.requires_grad = not task0_mode

    outputs = model(x)[controller]
    loss = criterion(outputs, y)
    loss.backward()
    optimizer.step()

    # Monitor the parameter updates
    for name, p in model.named_parameters():
        if name in ['out_task0.weight', 'out_task1.weight']:
            print(f"Controller: {controller}")
            print(name, p)

Did I miss anything in the training procedure? Or the overall setup will not work?


title: how to determine anchor sizes for RetinaNet model?

body: I am trying to implement a RetinaNet model in pytorch for my custom dataset, however, i am little confused on how some of the hyper-parameters are chosen. For example, the model uses anchor size of [32, 64, 128, 256, 512], anchor ratio of [0.5, 1, 2] and anchor scale of [1, 1.25, 1.58], but how do we determine these numbers for our own datasets. What are some of the ways we can go about when selecting param for the anchor box generation part ?


title: Torch installed but repeat_interleave not found

body:
I have PyTorch installed.
import torch runs without error. However, the function torch.repeat_interleave() is not found:
x = torch.tensor([1, 2, 3])
x.repeat_interleave(2)

gives AttributeError: 'Tensor' object has no attribute 'repeat_interleave'
Why?


title: How to find the index of a tensor in a list?

body:
I want to find the index of the smallest tensor (by some key function) in a list li.
So I did min and afterwards li.index(min_el). My MWE suggests that somehow tensors don't work with index.

import torch
li=[torch.ones(1,1), torch.zeros(2,2)]
li.index(li[0])
0
li.index(li[1])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../local/lib/python2.7/site-packages/torch/tensor.py", line 330, in __eq__
    return self.eq(other)
RuntimeError: inconsistent tensor size at /b/wheel/pytorch-src/torch/lib/TH/generic/THTensorMath.c:2679

I can of course make my own index function that first checks for size and then element wise. E.g.

def index(list, element):
    for i,el in enumerate(list):
        if el.size() == element.size():
            diff = el - element
            if (1- diff.byte()).all():
               return i
    return -1

I was just wondering, why doesn't index work?
Is there maybe a smart way of doing this not by hand that I'm missing?


title: Using google cloud ml gpu on python 3.7

body:
I am trying to run a ML model on google cloud ML. I am using pytorch and want to use the GPU. Using the standard Python3.6 version installed on the Google cloud VM, I get an error described below and tried solving it by upgrading the Python version to Python 3.7, but this version does not recognize the GPU that comes with the Google cloud VM.

Whenever I run my code (which works when ran locally) on the Google cloud VM (based on Python3.6) I get the error

python: symbol lookup error: /home/julsoles/anaconda3/lib/python3.6/site-packages/torch/lib/libtorch_python.so: undefined symbol: PySlice_Unpack

Trying to find a solution online, I figured out that this was an issue with the version of Python 3.6 and the only solution was to upgrade my version of Python.

I was able to upgrade my version of Python to Python3.7 in the Google Cloud VM and can run code with this new version using the command Python3.7 file.py.
Now, the issue is that whenever I run code using this version of Python, the VM does not recognize the GPU that comes with the system. I get the error

  File "pred.py", line 75, in
      predict(model_list, test_dataset)   File "pred.py", line 28, in predict
      x = Variable(torch.from_numpy(x).float()).cuda()   File "/opt/anaconda3/lib/python3.7/site-packages/torch/cuda/init.py",
  line 161, in _lazy_init
      _check_driver()   File "/opt/anaconda3/lib/python3.7/site-packages/torch/cuda/init.py",
  line 75, in _check_driver
      raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled

Right now, the only solution I have found is to run my code just using cpu, but it is painstakingly slow. Is there any way to make Python3.7 recognize the GPU so that I can run my code using the GPU?

Thanks for your help!