MullOverThings

Useful tips for everyday

# How should we do input selection for the neural networks model?

## How should we do input selection for the neural networks model?

How can you do input selection with neural networks?

1. Inspect the Hinton diagram and remove the variable whose weights are closest to zero.
2. Re-estimate the neural network with the variable removed.
3. Continue with step 1 until a stopping criterion is met.

## How do neural networks deal with missing values?

Being creative, it is possible to model a simple missing data mechanism with a neural network. You can represent the boolean variable (like smoker, yes/no) by one input neuron, with encoded input 1 for smoker and −1 for non-smoker. Give the value 0 as input to this neuron when the smoker variable is missing.

## What is model selection in machine learning?

Model selection is the process of selecting one final machine learning model from among a collection of candidate machine learning models for a training dataset. Model selection is the process of choosing one of the models as the final model that addresses the problem.

## Is it possible to train a neural network with missing data?

A number of practical problems have missing data in the datasets. These missing data are sometimes indispensable for solving problems. Therefore, people cannot simply ignore these missing data in datasets. A naive way for dealing with missing values is to fill them with a constant or a mean of its class.

## Why is it important to quantify uncertainty in neural networks?

Properly quantifying uncertainty is important because we (as practitioners training the models) can’t be confident in the model’s ability to generalize if it assigns arbitrarily high confidence to garbage input. The 3-class classifier was trained on images of cats, dogs and wild animals taken from Kaggle that can be downloaded here.

## What’s the problem with overconfidence in neural networks?

The problem isn’t that I passed an inappropriate image, because models in the real world are passed all sorts of garbage. It’s that the model is overconfident about an image far away from the training data. Instead we expect a more uniform distribution over the classes.

## Is there a scaling factor for ReLU networks?

Essentially, they prove that for a given class k, there exists a scaling factor α > 0 such that the softmax value of input α x as α → ∞ is equal to 1. This means that there are infinitely many inputs that obtain arbitrarily high confidence in ReLU networks.