Efficient Neural Network Inference for Resource Constrained Devices