Data Science Articles 1
Thank you for your question about Data Science articles published in the last two weeks. The short version is that data science article published in the last two weeks includes: "New Theory Cracks Open the Black Box of Deep Learning", "Intel Unveils Neuromorphic, Self-Learning Chip Codenamed Loihi", "Improving Efficiency in Convolutional Neural Network with Multilinear Filters", "The Consciousness Prior", and "Biosignals Learning and Synthesis Using Deep Neural Networks". Below you will find a deep dive into this answer.
I searched extensively for articles on data science (Deep Learning, Deep Architectures, Reinforcement Learning, Convolutional Neural Networks or anything with convolutional in it) published in the last two weeks. I excluded any article published in that timeframe that was introductory in nature or targeted at beginners, focusing on those that either explained recent developments or provided a unique view on already existing ones. I also excluded topics on semantics, natural language processing, LSTM (long short term networks), and Human Vision as requested. Many of the resources I searched had articles on the same subject under different headlines, so I only selected one of such stories discussing the same subject matter.
1) "New Theory Cracks Open the Black Box of Deep Learning"
The article, published on September 24, 2017, explains how a new idea dubbed information bottleneck helps to explain the reason artificial intelligence algorithms are very successful. The article explains that although machines' deep neural networks have learned to drive cars, converse, paint pictures, beat video games champions, they have often confounded their human creators who didn't expect their "deep learning" machine to be so good. However, "Naftali Tishby, a computer scientist and neuroscientist from the Hebrew University of Jerusalem, presented evidence in support of a new theory explaining how deep learning works. Tishby argues that deep neural networks learn according to a procedure called the “information bottleneck,” which he and two collaborators first described in purely theoretical terms in 1999. The idea is that a network rids noisy input data of extraneous details as if by squeezing the information through a bottleneck, retaining only the features most relevant to general concepts."
2) "Intel Unveils Neuromorphic, Self-Learning Chip Codenamed Loihi"
This article published on September 27 discusses Intel's newly launched self-learning chip, codenamed Loihi. The launch itself was announced by Intel on the 25th of September. Loihi is designed for AI and deep learning workloads. According to Dr. Michael Mayberry, "Loihi does not need to be trained in the traditional way and that it takes a new approach to this type of computing by using asynchronous spiking. Unlike a transistor, neurons do not constantly flip back and forth between a 0 and a 1. They trigger when signal thresholds are reached, and continue to fire so long as the number of spikes exceeds a given threshold."
3) "'Self learning' Intel chips glimpsed, Nvidia emits blueprints, AMD and Tesla rumors, and more"
This article published on September 28 is a roundup of developments in the AI field in the past week. The report covers the release of Intel's Loihi chip as well as that Nvidia announced something similar, called "the Nvidia Deep Learning Accelerator, an open-source set of Verilog blueprints for creating your own inference hardware accelerators for deep learning." In addition, Nvidia announced the launch of "TensorRT 3, software that performs "3.5x faster inference" on its latest Tesla V100 chips compared to its older P100 family. It supports optimization for models trained in Google's TensorFlow and Facebook's Caffe." Another interesting announcement covered in the report is that "Google Cloud Platformhttps://medium.com/towards-data-science/building-a-toy-detector-with-tensorflow-object-detection-api-63c0fdf2ac95 announced it has deployed Nvidia P100 GPUs in beta, and Nvidia K80 GPU accelerators for its Google Compute Engine, improving its lineup of on-demand hardware acceleration for AI." However, the article also covers older events such as Apple's announcement of its devices "real-time recognition of handwritten Chinese characters spanning a large inventory of 30,000 characters," with the aid of deep learning.
4) "Building a Toy Detector with Tensorflow Object Detection API"
This article is an update on the Priya Dwivedi popular project "Is Google Tensorflow Object Detection API the easiest way to implement image recognition?" In the original experiment, she used "the models provided by Tensorflow to detect common objects in youtube videos. These models were trained on the COCO dataset and work well on the 90 commonly found objects included in this dataset." In this update, she extended "the API to train on a new object that is not part of the COCO dataset. In this case, [she] chose a toy that was lying around." She reports that thus far, she is impressed by the ability of Tensorflow.
This is a research paper published by Prof. Yoshua Bengio published on the 26th of September. It is a technical paper that proposes a new "representation learning, which can be combined with other priors in order to help to disentangle abstract factors from each other." According to the author, "instead of making predictions in the sensory (e.g. pixel) space, the consciousness prior allow the agent to make predictions in the abstract space, with only a few dimensions of that space being involved in each of these predictions. The consciousness prior also makes it natural to map conscious states to natural language utterances or to express classical AI knowledge in
the form of facts and rules, although the conscious states may be richer than what can be expressed easily in the form of a sentence, a fact or a rule."
6) "Google’s AI chief thinks reports of the AI apocalypse are greatly exaggerated"
This article was published on September 19. It reports the view of John Giannandrea from Google that people are unnecessarily scared about the general purpose of AI. According to him, “I think there’s a huge amount of hype around AI right now. There’s a lot of people that are unreasonably concerned around the rise of general AI.” He goes on to say that "Machine learning and artificial intelligence are extremely important and will revolutionize our industry. What we’re doing is building tools like the Google search engine and making you more productive.” Unlike Elon Musk, he said he is not worried about AI apocalypse and that he objects "to the hype and soundbites that some people are making."
7) "Modeling the Resource Requirements of Convolutional Neural Networks (CNNs) on Mobile Devices"
This is a study published on the 27th of September. The study was conducted by Zongqing Lu, Swati Rallapalli, Kevin Chan, and Thomas La Porta. According to the authors, although Convolutional Neural Networks (CNNs) have revolutionized the research in computer vision, their complex pattern means that they are more suited to computer servers. The authors carried out the study with the aim of understanding "the resource requirements (time, memory) of CNNs on mobile devices." According to them, "by deploying several popular CNNs on mobile CPUs and GPUs, we measure and analyze the performance and resource usage for every layer of the CNNs. Our findings point out the potential ways of optimizing the performance on mobile devices. Second, we model the resource requirements of the different CNN computations. Finally, based on the measurement, profiling, and modeling, we build and evaluate our modeling tool, Augur, which takes a CNN configuration (descriptor) as the input and estimates the compute time and resource usage of the CNN, to give insights about whether and how efficiently a CNN can be run on a given mobile platform."
7) "Improving Efficiency in Convolutional Neural Network with Multilinear Filters"
The paper was published on 28 September by Dat Thanh Tran, Alexandros Iosifidis, and Moncef Gabbouj. The paper is about an alternative way of improving the efficiency in Convolutional Network. According to the authors, "instead of compressing a pre-trained network, in this work, we propose a generic neural network layer structure employing multilinear projection as the primary feature extractor. The proposed architecture requires several times less memory as compared to the traditional Convolutional Neural Networks (CNN), while inherits the similar design principles of a CNN. In addition, the proposed architecture is equipped with two computation schemes that enable computation reduction or scalability. Experimental results show the effectiveness of our compact projection that outperforms traditional CNN, while requiring far fewer parameters."
8) "Biosignals Learning and Synthesis Using Deep Neural Networks"
This paper was published on September 25. It is about "creation of novel algorithms for signal reconstruction in heavily noisy data and source detection in the biomedical engineering field." The authors of the paper are David Belo, João Rodrigues, João R. Vaz, Pedro Pezarat-Correia, and Hugo Gamboa. The authors explored "the gated recurrent units (GRU) employed in the training of respiration (RESP), electromyograms (EMG) and electrocardiograms (ECG)," and built a model based on the signals. According to the authors, "during the learning process, after a set of iterations, the model starts to grasp the basic morphological characteristics of the signal and later their cyclic characteristics. After training, these models’ predictions are closer to the signals that trained them, especially the RESP and ECG. This synthesis mechanism has shown relevant results that inspire the use to characterize signals from other physiological sources."
9) "AR and Deep Neural Networks Collide to Provide ModiFace"
This is an article published on September 28 announcing "a new live video-based hair tracking and hair color simulation technology utilizing a deep neural network architecture." According to Parham Aarabi, CEO of ModiFace and Professor at the University of Toronto, “we have been working on deep learning architectures for a long time now, and recent advances in both the neural network architectures, basic hardware level optimizations, as well as the availability of significant training data, have made photo-realistic video hair tracking and coloration possible.”
10) "Connectivity Learning in Multi-Branch Networks"
This article was published on the 27th of September by Karim Ahmed and Lorenzo Torresani. According to the authors, due to the "complexity of design choices in multi-branch architectures, prior work has adopted simple strategies, such as a fixed branching factor, the same input being fed to all parallel branches, and an additive combination of the outputs produced by all branches at aggregation points." In this work, the authors removed "these predefined choices and propose an algorithm to learn the connections between branches in the network. Instead of being chosen a priori by the human designer, the multi-branch connectivity is learned simultaneously." The authors demonstrate that their "approach to the problem of multiclass image classification using three different datasets where it yields consistently higher accuracy compared to the state-of-the-art “ResNeXt” multi-branch network given the same learning capacity with the weights of the network by optimizing a single loss function defined with respect to the end task."
11) "Learning to Inpaint for Image Compression"
This paper was published on 27 September. The authors studied "the design of deep architectures for lossy image compression." The authors of the paper are Mohammad Haris Baig, Vladlen Koltun, and Lorenzo Torresani. The authors presented "two architectural recipes in the context of multi-stage progressive encoders and empirically demonstrate their importance on compression performance." According to them, their work showed that: "(a) predicting the original image data from residuals in a multi-stage progressive architecture facilitates learning and leads to improved performance at approximating the original content and (b) learning to inpaint (from neighboring image pixels) before performing compression reduces the amount of information that must be stored to achieve a high-quality approximation. Incorporating these design choices in a baseline progressive encoder yields an average reduction of over 60% in file size with the similar quality compared to the original residual encoder."
To wrap it up, some of the articles published on deep learning in the last two weeks includes: "New Theory Cracks Open the Black Box of Deep Learning", "Intel Unveils Neuromorphic, Self-Learning Chip Codenamed Loihi", "Improving Efficiency in Convolutional Neural Network with Multilinear Filters", "The Consciousness Prior" and "Biosignals Learning and Synthesis Using Deep Neural Networks". Thank you for using Wonder! Please let us know if we can help with anything else!