Features
Features:

Product Tour >

Edraw AI >

Paid Plans:

Individuals >

Business >

Eduaction >
Resources
Blog

History

How-tos & Tips

Discovery

Biography

Business Analysis

Examples

AI concept Map

Free AI Mind Map Generator

Onenote Mind Map

Bcg Matrix Examples

Nike Marketing Strategy

Unilever SWOT Analysis

Make Mind Maps in Google Docs

Guide

FAQs

What's New

Resource Center
Templates
All Templates

Brain Storming Templates

Strategy and Planning Templates

Project Management Templates

Product Management Templates

Human Resources Templates

Agile Workflow Templates

Marketing Templates

Education Templates

Fun and Games Templates

User Gallery
Download
Pricing
Enterprise

MindMap Gallery data folder and model folder

data folder and model folder

This is a mind map about the data folder and model folder. The data folder and model folder are respectively responsible for storing data and model-related files in the project, and are an important part of project management and operation.

Edited at 2024-02-08 15:42:24

PlotWizard

Recent works View more works>>

data folder and model folder

PlotWizard

Recent works View more works>>

Recommended to you
Outline

central theme

data folder

Cub

Split

Test.csv

test set

Train.csv

Training set

Val.csv

Validation set

Miniimagenet

Split

Get_unlabel.py

Define image category list

Extract data from three CSV files (train.csv, val.csv and test.csv) related to the Mini-ImageNet dataset, collating the file names and category labels into a dictionary (mini_info) with the category labels sorted in order

Read data from three CSV files in the tieredimagenet/split directory and organize the filenames and category labels into another dictionary (tiered_info)

Categories that appear in the tiered_class_list but not in the mini_class_list are identified and these unlabeled categories are stored in the list unlabel. Then, print the number of unlabeled categories and their contents

Write unlabeled information into a CSV file to form unlabel.csv file

original data set

train.csv

val.csv

test.csv

Generated dataset

unlabel.csv

Additional testing and validation data

Aux_test.csv

Aux_val.csv

Tieredimagenet

Split

Get_spilt.py

Traverse different parts of the data set (train, val, test), generate corresponding CSV files, and write the file path and label into the CSV file for subsequent data processing, model training and other tasks.

Generated dataset

train.csv

val.csv

test.csv

model folder

Dataloader

Cub.py

A PyTorch dataset class CUB for loading the CUB (Caltech-UCSD Birds-200-2011) dataset.

Defines constants such as data set image path, segmentation path and cache path.

Data preprocessing function get_transforms:

Generate transformations required for data transformation based on input parameters, including random cropping, random horizontal flipping, color perturbation, and normalization.

Set different data normalization according to different backbone types to adapt to different model training needs.

Dataset class CUB:

Inherits PyTorch's Dataset class and is used to load CUB datasets.

The following operations are performed in the initialization function __init__: parse the CSV file and obtain data and labels.

Count the number of categories in a dataset.

Call the get_transforms method to obtain the transformations required for data transformation.

The parse_csv method is used to parse CSV files and obtain data and label information.

The __len__ method returns the length of the data set.

The __getitem__ method is used to get the sample at the specified index, which includes: trying to use the jpeg4py library to decode the JPEG image, and if that fails, using the PIL library to open the image.

Apply data transformation (preprocessing) and return images and labels.

Mini_imagenet.py

Create a PyTorch dataset class (MiniImageNet class) for working with the MiniImageNet dataset. This dataset class is used to load image data and provide corresponding data enhancement and preprocessing operations for training neural network models.

Data enhancement function get_transforms: Returns the transformation sequence of two image transformation methods (enhanced and non-enhanced) based on the specified image size, neural network structure (backbone), and data enhancement intensity

MiniImageNet data set class MiniImageNet: includes functions such as data loading, image path analysis, and data enhancement.

__init__ method: Initialize the data set class, accept the name of the data set division and some parameters, and call the get_transforms function to obtain the data conversion method.

parse_csv method: Parses image paths and labels from CSV files for building datasets.

__len__ method: Returns the length of the dataset (number of images).

__getitem__ method: Get the data item according to the index, try to use the jpeg4py library to decode the JPEG format image, if it fails, use the PIL library.

Samplers.py

For meta-learning and classification tasks, the code implements different sampler classes to provide flexible data sampling and loading methods for meta-learning tasks and a flexible data sampling and loading framework for meta-learning tasks. This makes it easier for researchers and developers to implement and test different meta-learning algorithms.

CategoriesSampler class:

During initialization, pass in the label (label), the number of samples in each batch (n_batch), the number of samples in each category (n_cls), and the number of samples sampled in each batch of each category (n_per).

Generate each batch of data through the __iter__ method, randomly select n_per samples for each category, and return a flattened one-dimensional tensor.

CategoriesViewSampler class:

During initialization, pass in the label (label), the number of samples in each batch (n_batch), the number of samples in each category (n_cls), the number of samples in each category for the support set (n_shot), and the number of samples in each category for query. The number of samples in the set (n_query) and the number of views in each task (n_view).

The __iter__ method generates each batch of data. For each category, n_shot n_query samples are randomly selected, of which the first n_shot are used for the support set and the last n_query are used for the query set. Support sets are labeled with additional information indicating the view number.

RandomSampler class:

During initialization, pass in the label (label), the number of samples in each batch (n_batch), and the number of labels per sample in each batch (n_per).

The __iter__ method generates data for each batch, and n_per samples are randomly selected for each batch.

ClassSampler class:

When initializing, pass in the label (label) and the number of samples per batch (n_per) for each category.

The __iter__ method generates data for each batch, and for each category, n_per samples are randomly selected.

InSetSampler class:

Used for ResNet Fine-Tune, the number of samples in each batch (n_batch), the number of labels for each sample in each batch (n_sbatch), and the sample pool (pool) are passed in during initialization.

The __iter__ method generates data for each batch, and each batch randomly selects n_sbatch samples from the sample pool.

Tiered_imagenet_raw.py

Mainly used to load the Tiered ImageNet data set, providing data preprocessing, image loading and label processing functions

Models

_init_.py

If a directory contains a file named __init__.py, Python will treat the directory as a package, allowing you to use the import statement to import modules or subpackages in the directory.

Tells Python to also import the FewShotModel class in the model.models.base module when importing the current package.

Maml.py

The Model-Agnostic Meta-Learning (MAML) framework implemented using PyTorch is used for meta-learning, implements MAML models, and provides different evaluation methods, including permutation evaluation and integrated evaluation. This is a general framework for meta-learning, applicable to a variety of tasks.

Section 1: Importing libraries and modules

Paragraph 2: update_params function

This function accepts a loss value, a dictionary of parameters (params), and updates the parameters using gradient descent. The function returns the updated parameters.

Paragraph 3: inner_train_step function

The inner_train_step function represents the inner training step in meta-learning. It accepts a model, supporting data and some parameters, then performs internal training and returns updated parameters after internal training.

Paragraph 4: MAML classes

Performs a forward pass of MAML, using internally trained parameters. There are other methods for evaluation such as forward_eval, forward_eval_perm and forward_eval_ensemble.

Paragraph 5: Model implementation

Select ResNetMAML as the backbone network according to the given backbone network category and initialize the model parameters. The update of parameters is performed in inner_train_step, using the cross-entropy loss function for internal training. The prediction results are scaled using the temperature parameter.

MAMLUnicorn.py

Implemented a meta-learning algorithm, specifically a variant based on Model-Agnostic Meta-Learning (MAML), for Few-Shot learning tasks

Networks

Convnet.py

Defines a basic convolutional neural network (ConvNet) model

Its main function is to process the input image through convolution and pooling operations, and finally output a feature tensor with z_dim channels, which can be used for various tasks, such as image classification, feature extraction, etc.

Convnet_maml.py

A model based on meta-learning is implemented, specifically the MAML (Model-Agnostic Meta-Learning) algorithm for meta-learning on small sample learning tasks.

Dropblock.py

Defines a PyTorch module called DropBlock that implements a regularization technique called DropBlock

Res12.py

Neural network designed based on ResNet architecture, mainly used for Few-Shot Learning tasks

The entire network adopts the structure of ResNet, but the design takes into account the characteristics of the Few-Shot Learning task, including the introduction of DropBlock and the dynamic adjustment of some parameters. At the same time, the code also contains initialization operations for model parameters.

Res12_maml.py

A neural network based on ResNet architecture, mainly used for model adaptability (Model-Agnostic Meta-Learning, MAML) and sharing of model parameters in meta-learning (Meta-Learning).