## Fully connected neuron network

Traditional NN

- The weight matrix A is
*N*by*M*so that the network is “fully connected”. - All nodes on adjacent layers are fully connected with each other
- Can be seen as with M
*“*kernels” which has*N*dimensions each - Many parameters; suffer severe overfitting

## Locally connected neural network

- Output is based only on the “receptive field” of size
*P,*so weight matrix*W*is*P*by*M*(P<N) *M*kernels each with dimension*P*- Less parameters to train, less overfit

## Convolutional Neural Network (shared-weight local)

**Shared****weight**local connected neural network- Weight matrix
*W*is*P*by 1 (since we can write*P*as a Matrix as well) - Even fewer parameters to train
- Can simultaneously train many
**maps**to extract more features

Advertisements