Pytoch -- data loading and processing

Pytorch A tool for data processing in

scikit-image： For images IO And transformation
pandas： Used to make it easier to csv analysis
from here Download datasets .
The data is stored in “data / faces /” A directory of . This dataset is actually imagenet Dataset standard
Note is face In the picture of dlib Face detection (dlib’s pose estimation) A good picture . What we need to deal with is a data set of facial posture . Here's the picture

Dataset class
torch.utils.data.Dataset An abstract class that represents a dataset .
Custom datasets should inherit Dataset And cover the following methods * __len__ Realization len(dataset) Return the size of the dataset . * __getitem__ Used to get some index data , for example dataset[i] Medium (i).

Create dataset classes
Create a dataset class for the facial dataset . We will be in __init__ Read from csv File contents of , stay __getitem__ Read pictures from .
Our data samples will be based on such a dictionary {‘image’: image, ‘landmarks’: landmarks} organization .
Our dataset class will add a Optional parameters transform To facilitate the pretreatment of samples . init The method is shown in the figure below ：

class FaceLandmarksDataset(Dataset):
    """ Facial marker dataset ."""
    
    def __init__(self, csv_file, root_dir, transform=None):
        """ csv_file（string）： Annotated csv Path to file . root_dir（string）： Directory containing all images . transform（callable, optional）： Optional transformations available on a sample  """
        self.landmarks_frame = pd.read_csv(csv_file)
        self.root_dir = root_dir
        self.transform = transform
    
    def __len__(self):
        return len(self.landmarks_frame)

    def __getitem__(self, idx):
        img_name = os.path.join(self.root_dir,self.landmarks_frame.iloc[idx, 0])
        image = io.imread(img_name)
        landmarks = self.landmarks_frame.iloc[idx, 1:]
        landmarks = np.array([landmarks])
        landmarks = landmarks.astype('float').reshape(-1, 2)
        sample = {
    'image': image, 'landmarks': landmarks}
    
        if self.transform:
            sample = self.transform(sample)

        return sample

Data visualization

face_dataset = FaceLandmarksDataset(csv_file='data/faces/face_landmarks.csv',root_dir='data/faces/')

fig = plt.figure()

for i in range(len(face_dataset)):
    sample = face_dataset[i]
    
    print(i, sample['image'].shape, sample['landmarks'].shape)
    
    ax = plt.subplot(1, 4, i + 1)
    plt.tight_layout()
    ax.set_title('Sample #{}'.format(i))
    ax.axis('off')
    show_landmarks(**sample)
    
    if i == 3:
        plt.show()
        break

Data transformation
Through the above example, we will find that the picture is not the same size .
Most neural networks assume that the size of the image is the same .
So we need to do some pretreatment . Let's create three transformations :
* Rescale ： Zoom in and out
* RandomCrop ： Randomly crop the picture .
* ToTensor ： This is a data enhancement operation , hold numpy Format picture to torch Format picture ( We need to exchange axes ).
We will write them in the form of callable classes instead of simple functions , So you don't need to pass parameters every time you call . We just need to achieve call Method , have to When necessary init Method . We can call these transformations like this :