How Deep Neural Networks perform Image Classification

Author: Charter Global
Published: January 15, 2020
Categories: Machine Learning

Artificial Intelligence and Machine Learning are becoming exponentially more popular in the business world – especially when used as tools to better meet business objectives. Through the process of deep learning neural networks, AI and ML have the ability to classify images.  A common and highly effective approach to deep learning on small image datasets is to use a pre-trained network like VGG16 and RESNET. One can also build custom deep learning models for image classification.


There are several popular Deep Learning Frameworks, including:


  1. Tensor Flow/Keras
  2. Pytorch
  3. CNTK (Cognitive Toolkit)
  4. Sage Maker


Teaching Computers How to Recognize Objects: 


When implementing the right tools, we can empower our computers with the ability of recognition, where they can virtually distinguish images from one another. The popular CIFAR-10 Dataset is representative of this ability, pictured below. Each picture has been assigned a label corresponding to the category of image class. The images and labels are grouped according to the “training data,” or the classification of the various models. The training data is assigned to an image upon its discovery, providing a label for the computer to recognize moving forward


CIFAR 10 Dataset



How a Deep Neural Network Interprets and Classifies an Image


A deep neural network learns the features of an image by recognizing certain attributes, or layers illustrated below.  Upon recognition of the image or the input, the network first analyzes the simple elements of the photo, like the outermost edges. Then, an analysis is performed of the corners and contours within the image, followed by a final assessment of the most complex features which identifies the various parts within.


Deep Neural Network Image Classification


Ecommerce Web Services– online Self-service Use Case


Online self-service is essential for businesses that rely on eCommerce web services. For example, in the plumbing industry, one of the major challenges customers face is purchasing a spare part for their household plumbing repairs.  The customer, or end-user, may not know the product code (or PLU#) necessary to place the order for the part they need. While customer care agents can assist with these transactions, it’s best if the customer can simply identify the part by relying on the artificial intelligence inherent in their computer’s deep neural network, exhibited in the case study below.


Problem: An online shopper does not have the right PLU# associated with the part needed to repair a plumbing issue in their home. The only frame of reference the customer has is a picture of the part they have uploaded.


Solution: The build artificial intelligence (or deep neural network) image matching module compares the given image from the customer with images of all cataloged products, identifying the most accurate match for their consideration.




  • Improves end-user experience and customer satisfaction.
  • Reduces manual intervention (by customer care agents) in identifying the right spare part
  • It provides hassle-free customer service.


Model development – Practical Tips


Here are some tips for customizing the model to work for you.


Model Development Tips


Model performance and Improvement process


Is the model performing poorly? Identify defects within the model’s performance as a result of overfitting, underfitting, or defective software.




Underfitting occurs when the model unable to fit within the parameters of the training data (i.e. improper image settings), resulting in a high error rate on both the training data and unseen test data. As a result, the difference between error rates is less.


When overfitting occurs, the model perfectly fits the training data, and can properly identify training data, but fails on unseen data. So the low error rate is observed on training data and a high error rate is observed on unseen test data. The resulting difference between error rates will be very high.


Model Complexity and Prediction Errors


Model Improvement Process


Ensure your objectives are met by monitoring and applying incremental changes such as gathering new data, adjusting hyperparameters, or changing algorithms.