A talk given at Computational Neuroscience Seminar @ University of Tartu. We discuss the idea behind deep neural network that has won ILSVRC (ImageNet) 2015 and COCO 2015 Image competitions.
Deep residual learning for image recognitionYoonho Shin
This document presents a deep residual learning framework for training very deep neural networks for image recognition tasks. The framework addresses the degradation problem that occurs when networks become substantially deeper by introducing identity shortcut connections that skip one or more layers. Experiments show that residual networks are able to train substantially deeper, over 100 layers deep, and achieve state-of-the-art results on image classification and other computer vision tasks compared to traditional networks. The residual learning framework allows networks to learn residual functions with reference to the layer inputs rather than unreferenced functions.
Residual neural networks (ResNets) solve the vanishing gradient problem through shortcut connections that allow gradients to flow directly through the network. The ResNet architecture consists of repeating blocks with convolutional layers and shortcut connections. These connections perform identity mappings and add the outputs of the convolutional layers to the shortcut connection. This helps networks converge earlier and increases accuracy. Variants include basic blocks with two convolutional layers and bottleneck blocks with three layers. Parameters like number of layers affect ResNet performance, with deeper networks showing improved accuracy. YOLO is a variant that replaces the softmax layer with a 1x1 convolutional layer and logistic function for multi-label classification.
ResNet basics (Deep Residual Network for Image Recognition)Sanjay Saha
The document presents the deep residual learning framework proposed in the paper "Deep Residual Learning for Image Recognition". The framework aims to make it easier to optimize extremely deep convolutional neural networks. It does this by introducing "skip connections" that allow layers to learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. This addresses issues like vanishing gradients in very deep networks. The authors demonstrate that residual networks are easier to optimize and can gain accuracy from increased depth, outperforming standard networks.
GANs are the new hottest topic in the ML arena; however, they present a challenge for the researchers and the engineers alike. Their design, and most importantly, the code implementation has been causing headaches to the ML practitioners, especially when moving to production.
Starting from the very basic of what a GAN is, passing trough Tensorflow implementation, using the most cutting-edge APIs available in the framework, and finally, production-ready serving at scale using Google Cloud ML Engine.
Slides for the talk: https://www.pycon.it/conference/talks/deep-diving-into-gans-form-theory-to-production
Github repo: https://github.com/zurutech/gans-from-theory-to-production
In this presentation we discuss the convolution operation, the architecture of a convolution neural network, different layers such as pooling etc. This presentation draws heavily from A Karpathy's Stanford Course CS 231n
The document summarizes the Batch Normalization technique presented in the paper "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift". Batch Normalization aims to address the issue of internal covariate shift in deep neural networks by normalizing layer inputs to have zero mean and unit variance. It works by computing normalization statistics for each mini-batch and applying them to the inputs. This helps in faster and more stable training of deep networks by reducing the distribution shift across layers. The paper presented ablation studies on MNIST and ImageNet datasets showing Batch Normalization improves training speed and accuracy compared to prior techniques.
Transfer Learning and Fine-tuning Deep Neural NetworksPyData
This document outlines Anusua Trivedi's talk on transfer learning and fine-tuning deep neural networks. The talk covers traditional machine learning versus deep learning, using deep convolutional neural networks (DCNNs) for image analysis, transfer learning and fine-tuning DCNNs, recurrent neural networks (RNNs), and case studies applying these techniques to diabetic retinopathy prediction and fashion image caption generation.
Deep residual learning for image recognitionYoonho Shin
This document presents a deep residual learning framework for training very deep neural networks for image recognition tasks. The framework addresses the degradation problem that occurs when networks become substantially deeper by introducing identity shortcut connections that skip one or more layers. Experiments show that residual networks are able to train substantially deeper, over 100 layers deep, and achieve state-of-the-art results on image classification and other computer vision tasks compared to traditional networks. The residual learning framework allows networks to learn residual functions with reference to the layer inputs rather than unreferenced functions.
Residual neural networks (ResNets) solve the vanishing gradient problem through shortcut connections that allow gradients to flow directly through the network. The ResNet architecture consists of repeating blocks with convolutional layers and shortcut connections. These connections perform identity mappings and add the outputs of the convolutional layers to the shortcut connection. This helps networks converge earlier and increases accuracy. Variants include basic blocks with two convolutional layers and bottleneck blocks with three layers. Parameters like number of layers affect ResNet performance, with deeper networks showing improved accuracy. YOLO is a variant that replaces the softmax layer with a 1x1 convolutional layer and logistic function for multi-label classification.
ResNet basics (Deep Residual Network for Image Recognition)Sanjay Saha
The document presents the deep residual learning framework proposed in the paper "Deep Residual Learning for Image Recognition". The framework aims to make it easier to optimize extremely deep convolutional neural networks. It does this by introducing "skip connections" that allow layers to learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. This addresses issues like vanishing gradients in very deep networks. The authors demonstrate that residual networks are easier to optimize and can gain accuracy from increased depth, outperforming standard networks.
GANs are the new hottest topic in the ML arena; however, they present a challenge for the researchers and the engineers alike. Their design, and most importantly, the code implementation has been causing headaches to the ML practitioners, especially when moving to production.
Starting from the very basic of what a GAN is, passing trough Tensorflow implementation, using the most cutting-edge APIs available in the framework, and finally, production-ready serving at scale using Google Cloud ML Engine.
Slides for the talk: https://www.pycon.it/conference/talks/deep-diving-into-gans-form-theory-to-production
Github repo: https://github.com/zurutech/gans-from-theory-to-production
In this presentation we discuss the convolution operation, the architecture of a convolution neural network, different layers such as pooling etc. This presentation draws heavily from A Karpathy's Stanford Course CS 231n
The document summarizes the Batch Normalization technique presented in the paper "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift". Batch Normalization aims to address the issue of internal covariate shift in deep neural networks by normalizing layer inputs to have zero mean and unit variance. It works by computing normalization statistics for each mini-batch and applying them to the inputs. This helps in faster and more stable training of deep networks by reducing the distribution shift across layers. The paper presented ablation studies on MNIST and ImageNet datasets showing Batch Normalization improves training speed and accuracy compared to prior techniques.
Transfer Learning and Fine-tuning Deep Neural NetworksPyData
This document outlines Anusua Trivedi's talk on transfer learning and fine-tuning deep neural networks. The talk covers traditional machine learning versus deep learning, using deep convolutional neural networks (DCNNs) for image analysis, transfer learning and fine-tuning DCNNs, recurrent neural networks (RNNs), and case studies applying these techniques to diabetic retinopathy prediction and fashion image caption generation.
AlexNet achieved unprecedented results on the ImageNet dataset by using a deep convolutional neural network with over 60 million parameters. It achieved top-1 and top-5 error rates of 37.5% and 17.0%, significantly outperforming previous methods. The network architecture included 5 convolutional layers, some with max pooling, and 3 fully-connected layers. Key aspects were the use of ReLU activations for faster training, dropout to reduce overfitting, and parallelizing computations across two GPUs. This dramatic improvement demonstrated the potential of deep learning for computer vision tasks.
This document provides an agenda for a presentation on deep learning, neural networks, convolutional neural networks, and interesting applications. The presentation will include introductions to deep learning and how it differs from traditional machine learning by learning feature representations from data. It will cover the history of neural networks and breakthroughs that enabled training of deeper models. Convolutional neural network architectures will be overviewed, including convolutional, pooling, and dense layers. Applications like recommendation systems, natural language processing, and computer vision will also be discussed. There will be a question and answer section.
We trained a large, deep convolutional neural network to classify the 1.2 million
high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif-
ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5%
and 17.0% which is considerably better than the previous state-of-the-art. The
neural network, which has 60 million parameters and 650,000 neurons, consists
of five convolutional layers, some of which are followed by max-pooling layers,
and three fully-connected layers with a final 1000-way softmax. To make train-
ing faster, we used non-saturating neurons and a very efficient GPU implemen-
tation of the convolution operation. To reduce overfitting in the fully-connected
layers we employed a recently-developed regularization method called “dropout”
that proved to be very effective. We also entered a variant of this model in the
ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%,
compared to 26.2% achieved by the second-best entry.
Deep learning for image super resolutionPrudhvi Raj
Using Deep Convolutional Networks, the machine can learn end-to-end mapping between the low/high-resolution images. Unlike traditional methods, this method jointly optimizes all the layers of the image. A light-weight CNN structure is used, which is simple to implement and provides formidable trade-off from the existential methods.
http://imatge-upc.github.io/telecombcn-2016-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
Summary:
There are three parts in this presentation.
A. Why do we need Convolutional Neural Network
- Problems we face today
- Solutions for problems
B. LeNet Overview
- The origin of LeNet
- The result after using LeNet model
C. LeNet Techniques
- LeNet structure
- Function of every layer
In the following Github Link, there is a repository that I rebuilt LeNet without any deep learning package. Hope this can make you more understand the basic of Convolutional Neural Network.
Github Link : https://github.com/HiCraigChen/LeNet
LinkedIn : https://www.linkedin.com/in/YungKueiChen
Sogang University Machine Learning and Data Mining lab seminar, Neural Networks for newbies and Convolutional Neural Networks. This is prerequisite material to understand deep convolutional architecture.
Generative adversarial networks (GANs) use two neural networks, a generator and discriminator, that compete against each other. The generator aims to produce realistic samples to fool the discriminator, while the discriminator tries to distinguish real samples from generated ones. This adversarial training can produce high-quality, sharp samples but is challenging to train as the generator and discriminator must be carefully balanced.
This document discusses image-to-image translation using conditional adversarial networks. It introduces the problem of mapping pixels to pixels across different image representations. Generative adversarial networks are presented as a solution, using a generator network to synthesize images and a discriminator network to identify fakes. The document describes a U-Net architecture for the generator and a PatchGAN discriminator. It then shows experiments on tasks like semantic labeling, colorization, and discusses evaluation methods and failure cases.
A convolutional neural network (CNN, or ConvNet) is a class of deep, feed-forward artificial neural networks that has Successfully been applied to analyzing visual imagery
The document discusses convolutional neural networks (CNNs). It begins with an introduction and overview of CNN components like convolution, ReLU, and pooling layers. Convolution layers apply filters to input images to extract features, ReLU introduces non-linearity, and pooling layers reduce dimensionality. CNNs are well-suited for image data since they can incorporate spatial relationships. The document provides an example of building a CNN using TensorFlow to classify handwritten digits from the MNIST dataset.
This document describes DenseNets, a type of convolutional neural network architecture. DenseNets connect each layer to every other layer in a feed-forward fashion to encourage feature reuse and consolidate feature maps early in the network. This architecture improves information and gradient flow. The document outlines key DenseNet concepts like collective knowledge, compression layers, and growth rate. It also provides results comparing DenseNets to ResNet on CIFAR-10 and ImageNet datasets.
ResNet is a neural network architecture used for computer vision tasks that allows for hundreds or thousands of layers. It addresses the problem of vanishing gradients that occurs when training very deep networks. ResNet uses skip connections to feed the activation from one layer directly into another layer deeper in the network. This allows information to skip layers and facilitates the training of very deep neural networks, enabling them to gather complex information from images.
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
In this project, we propose methods for semantic segmentation with the deep learning state-of-the-art models. Moreover,
we want to filterize the segmentation to the specific object in specific application. Instead of concentrating on unnecessary objects we
can focus on special ones and make it more specialize and effecient for special purposes. Furtheromore, In this project, we leverage
models that are suitable for face segmentation. The models that are used in this project are Mask-RCNN and DeepLabv3. The
experimental results clearly indicate that how illustrated approach are efficient and robust in the segmentation task to the previous work
in the field of segmentation. These models are reached to 74.4 and 86.6 precision of Mean of Intersection over Union. The visual
Results of the models are shown in Appendix part.
Convolutional neural networks (CNNs) are a type of neural network designed to process images. CNNs use a series of convolution and pooling layers to extract features from images. Convolution multiplies the image with filters to produce feature maps, while pooling reduces the size of the representation to reduce computation. This process allows the network to learn increasingly complex features from the input image and classify it. CNNs have applications in areas like facial recognition, document analysis, and image classification.
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
AI&BigData Lab 2016. Александр Баев: Transfer learning - зачем, как и где.GeeksLab Odessa
4.6.16 AI&BigData Lab
Upcoming events: goo.gl/I2gJ4H
Поговорим об одной из базовых практических техник обучения нейронных сетей - предобучение, finetuning, transfer learning. В каких случаях применять, какие модели использовать, где их брать и как адаптировать.
AlexNet achieved unprecedented results on the ImageNet dataset by using a deep convolutional neural network with over 60 million parameters. It achieved top-1 and top-5 error rates of 37.5% and 17.0%, significantly outperforming previous methods. The network architecture included 5 convolutional layers, some with max pooling, and 3 fully-connected layers. Key aspects were the use of ReLU activations for faster training, dropout to reduce overfitting, and parallelizing computations across two GPUs. This dramatic improvement demonstrated the potential of deep learning for computer vision tasks.
This document provides an agenda for a presentation on deep learning, neural networks, convolutional neural networks, and interesting applications. The presentation will include introductions to deep learning and how it differs from traditional machine learning by learning feature representations from data. It will cover the history of neural networks and breakthroughs that enabled training of deeper models. Convolutional neural network architectures will be overviewed, including convolutional, pooling, and dense layers. Applications like recommendation systems, natural language processing, and computer vision will also be discussed. There will be a question and answer section.
We trained a large, deep convolutional neural network to classify the 1.2 million
high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif-
ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5%
and 17.0% which is considerably better than the previous state-of-the-art. The
neural network, which has 60 million parameters and 650,000 neurons, consists
of five convolutional layers, some of which are followed by max-pooling layers,
and three fully-connected layers with a final 1000-way softmax. To make train-
ing faster, we used non-saturating neurons and a very efficient GPU implemen-
tation of the convolution operation. To reduce overfitting in the fully-connected
layers we employed a recently-developed regularization method called “dropout”
that proved to be very effective. We also entered a variant of this model in the
ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%,
compared to 26.2% achieved by the second-best entry.
Deep learning for image super resolutionPrudhvi Raj
Using Deep Convolutional Networks, the machine can learn end-to-end mapping between the low/high-resolution images. Unlike traditional methods, this method jointly optimizes all the layers of the image. A light-weight CNN structure is used, which is simple to implement and provides formidable trade-off from the existential methods.
http://imatge-upc.github.io/telecombcn-2016-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
Summary:
There are three parts in this presentation.
A. Why do we need Convolutional Neural Network
- Problems we face today
- Solutions for problems
B. LeNet Overview
- The origin of LeNet
- The result after using LeNet model
C. LeNet Techniques
- LeNet structure
- Function of every layer
In the following Github Link, there is a repository that I rebuilt LeNet without any deep learning package. Hope this can make you more understand the basic of Convolutional Neural Network.
Github Link : https://github.com/HiCraigChen/LeNet
LinkedIn : https://www.linkedin.com/in/YungKueiChen
Sogang University Machine Learning and Data Mining lab seminar, Neural Networks for newbies and Convolutional Neural Networks. This is prerequisite material to understand deep convolutional architecture.
Generative adversarial networks (GANs) use two neural networks, a generator and discriminator, that compete against each other. The generator aims to produce realistic samples to fool the discriminator, while the discriminator tries to distinguish real samples from generated ones. This adversarial training can produce high-quality, sharp samples but is challenging to train as the generator and discriminator must be carefully balanced.
This document discusses image-to-image translation using conditional adversarial networks. It introduces the problem of mapping pixels to pixels across different image representations. Generative adversarial networks are presented as a solution, using a generator network to synthesize images and a discriminator network to identify fakes. The document describes a U-Net architecture for the generator and a PatchGAN discriminator. It then shows experiments on tasks like semantic labeling, colorization, and discusses evaluation methods and failure cases.
A convolutional neural network (CNN, or ConvNet) is a class of deep, feed-forward artificial neural networks that has Successfully been applied to analyzing visual imagery
The document discusses convolutional neural networks (CNNs). It begins with an introduction and overview of CNN components like convolution, ReLU, and pooling layers. Convolution layers apply filters to input images to extract features, ReLU introduces non-linearity, and pooling layers reduce dimensionality. CNNs are well-suited for image data since they can incorporate spatial relationships. The document provides an example of building a CNN using TensorFlow to classify handwritten digits from the MNIST dataset.
This document describes DenseNets, a type of convolutional neural network architecture. DenseNets connect each layer to every other layer in a feed-forward fashion to encourage feature reuse and consolidate feature maps early in the network. This architecture improves information and gradient flow. The document outlines key DenseNet concepts like collective knowledge, compression layers, and growth rate. It also provides results comparing DenseNets to ResNet on CIFAR-10 and ImageNet datasets.
ResNet is a neural network architecture used for computer vision tasks that allows for hundreds or thousands of layers. It addresses the problem of vanishing gradients that occurs when training very deep networks. ResNet uses skip connections to feed the activation from one layer directly into another layer deeper in the network. This allows information to skip layers and facilitates the training of very deep neural networks, enabling them to gather complex information from images.
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
In this project, we propose methods for semantic segmentation with the deep learning state-of-the-art models. Moreover,
we want to filterize the segmentation to the specific object in specific application. Instead of concentrating on unnecessary objects we
can focus on special ones and make it more specialize and effecient for special purposes. Furtheromore, In this project, we leverage
models that are suitable for face segmentation. The models that are used in this project are Mask-RCNN and DeepLabv3. The
experimental results clearly indicate that how illustrated approach are efficient and robust in the segmentation task to the previous work
in the field of segmentation. These models are reached to 74.4 and 86.6 precision of Mean of Intersection over Union. The visual
Results of the models are shown in Appendix part.
Convolutional neural networks (CNNs) are a type of neural network designed to process images. CNNs use a series of convolution and pooling layers to extract features from images. Convolution multiplies the image with filters to produce feature maps, while pooling reduces the size of the representation to reduce computation. This process allows the network to learn increasingly complex features from the input image and classify it. CNNs have applications in areas like facial recognition, document analysis, and image classification.
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
AI&BigData Lab 2016. Александр Баев: Transfer learning - зачем, как и где.GeeksLab Odessa
4.6.16 AI&BigData Lab
Upcoming events: goo.gl/I2gJ4H
Поговорим об одной из базовых практических техник обучения нейронных сетей - предобучение, finetuning, transfer learning. В каких случаях применять, какие модели использовать, где их брать и как адаптировать.
Deep Learning Class #2 - Deep learning for Images, I See What You MeanHolberton School
Slides by Louis Monier (Altavista Co-Founder & CTO) for Deep Learning keynote #2 at Holberton School. http://www.meetup.com/Holberton-School/events/230547621/
The keynote was followed by a workshop prepared by Gregory Renard. If you want to assist to similar keynote for free, checkout http://www.meetup.com/Holberton-School/
This document discusses ConvNetJS and CaffeJS, which allow running convolutional neural networks in the browser. ConvNetJS defines layers and volumes to represent neural network structure and activations. It supports training networks with backpropagation. CaffeJS imports pre-trained models defined in Caffe's format and runs the forward pass in ConvNetJS. Challenges include slow performance, limited memory, and representing certain Caffe layers and structures. Future work could explore techniques like network in a network, fully convolutional networks, and WebAssembly compilation to improve browser deep learning.
Slides by Víctor Garcia about the paper:
Reed, Scott, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. "Generative adversarial text to image synthesis." ICML 2016.
Slides by Víctor Garcia about:
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A. Efros, "Image-to-Image Translation Using Conditional Adversarial Networks".
In arxiv, 2016.
https://phillipi.github.io/pix2pix/
We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without hand-engineering our loss functions either.
We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without hand-engineering our loss functions either.
https://telecombcn-dl.github.io/2017-dlsl/
Winter School on Deep Learning for Speech and Language. UPC BarcelonaTech ETSETB TelecomBCN.
The aim of this course is to train students in methods of deep learning for speech and language. Recurrent Neural Networks (RNN) will be presented and analyzed in detail to understand the potential of these state of the art tools for time series processing. Engineering tips and scalability issues will be addressed to solve tasks such as machine translation, speech recognition, speech synthesis or question answering. Hands-on sessions will provide development skills so that attendees can become competent in contemporary data analytics tools.
Miriam Bellver, Xavier Giro-i-Nieto, Ferran Marques, and Jordi Torres. "Hierarchical Object Detection with Deep Reinforcement Learning." In Deep Reinforcement Learning Workshop (NIPS). 2016.
We present a method for performing hierarchical object detection in images guided by a deep reinforcement learning agent. The key idea is to focus on those parts of the image that contain richer information and zoom on them. We train an intelligent agent that, given an image window, is capable of deciding where to focus the attention among five different predefined region candidates (smaller windows). This procedure is iterated providing a hierarchical image analysis.We compare two different candidate proposal strategies to guide the object search: with and without overlap. Moreover, our work compares two different strategies to extract features from a convolutional neural network for each region proposal: a first one that computes new feature maps for each region proposal, and a second one that computes the feature maps for the whole image to later generate crops for each region proposal. Experiments indicate better results for the overlapping candidate proposal strategy and a loss of performance for the cropped image features due to the loss of spatial resolution. We argue that, while this loss seems unavoidable when working with large amounts of object candidates, the much more reduced amount of region proposals generated by our reinforcement learning agent allows considering to extract features for each location without sharing convolutional computation among regions.
https://imatge-upc.github.io/detection-2016-nipsws/
This document proposes and describes Bayesian Dark Knowledge, a method to combine Stochastic Gradient Langevin Dynamics (SGLD) with knowledge distillation. SGLD allows learning of large-scale Bayesian models but requires storing multiple parameter copies. Distillation transfers knowledge from large teacher networks to a single student network. Bayesian Dark Knowledge trains a single student network using soft labels from an SGLD-trained ensemble of teachers, avoiding storing multiple parameter copies. The document discusses background on SGLD and distillation, and describes the Bayesian Dark Knowledge method, algorithm, and results. It concludes by discussing potential improvements to the original approach.
Deep Learning And Business Models (VNITC 2015-09-13)Ha Phuong
Deep Learning and Business Models
Tran Quoc Hoan discusses deep learning and its applications, as well as potential business models. Deep learning has led to significant improvements in areas like image and speech recognition compared to traditional machine learning. Some business models highlighted include developing deep learning frameworks, building hardware optimized for deep learning, using deep learning for IoT applications, and providing deep learning APIs and services. Deep learning shows promise across many sectors but also faces challenges in fully realizing its potential.
The document discusses attention mechanisms and their implementation in TensorFlow. It begins with an overview of attention mechanisms and their use in neural machine translation. It then reviews the code implementation of an attention mechanism for neural machine translation from English to French using TensorFlow. Finally, it briefly discusses pointer networks, an attention mechanism variant, and code implementation of pointer networks for solving sorting problems.
Improving Hardware Efficiency for DNN ApplicationsChester Chen
Speaker: Dr. Hai (Helen) Li is the Clare Boothe Luce Associate Professor of Electrical and Computer Engineering and Co-director of the Duke Center for Evolutionary Intelligence at Duke University
In this talk, I will introduce a few recent research spotlights by the Duke Center for Evolutionary Intelligence. The talk will start with the structured sparsity learning (SSL) method which attempts to learn a compact structure from a bigger DNN to reduce computation cost. It generates a regularized structure with high execution efficiency. Our experiments on CPU, GPU, and FPGA platforms show on average 3~5 times speedup of convolutional layer computation of AlexNet. Then, the implementation and acceleration of DNN applications on mobile computing systems will be introduced. MoDNN is a local distributed system which partitions DNN models onto several mobile devices to accelerate computations. ApesNet is an efficient pixel-wise segmentation network, which understands road scenes in real-time, and has achieved promising accuracy. Our prospects on the adoption of emerging technology will also be given at the end of this talk, offering the audiences an alternative thinking about the future evolution and revolution of modern computing systems.
Comparison of Learning Algorithms for Handwritten Digit RecognitionSafaa Alnabulsi
This document compares different machine learning algorithms for handwritten digit recognition on the MNIST dataset. Convolutional neural networks achieved the best results, with LeNet5 achieving 0.9% error and boosted LeNet4 achieving the lowest error rate of 0.7%. Neural networks required more training time but had faster recognition times and lower memory requirements compared to nearest neighbor classifiers. Overall, convolutional neural networks were best suited for handwritten digit recognition due to their ability to handle variations in size, position and orientation of digits.
Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networksareeasiertooptimize,andcangainaccuracyfrom considerably increased depth. On the ImageNet dataset we evaluate residualnets with adepth of up to152 layers—8× deeper than VGG nets [41] but still having lower complexity. Anensembleoftheseresidualnetsachieves3.57%error ontheImageNettestset. Thisresultwonthe1stplaceonthe ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residualnetsarefoundationsofoursubmissionstoILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
ResNet, short for "Residual Network," is a type of deep neural network architecture that was introduced by Microsoft researchers in 2015. ResNet is designed to address the problem of vanishing gradients, which can occur in deep neural networks that are many layers deep.
The main innovation in ResNet is the use of residual connections, also known as skip connections. These connections allow information from earlier layers of the network to bypass some of the later layers and be directly fed into the later layers. This helps to ensure that the gradient signal from the output can propagate back through the network during training, which can help to prevent the vanishing gradient problem.
ResNet has been shown to be very effective at image recognition and other computer vision tasks. It has achieved state-of-the-art performance on a number of benchmark datasets, such as ImageNet. Since its introduction, many variations and improvements to the original ResNet architecture have been proposed, including ResNeXt, Wide ResNet, and Residual Attention Network (RANet).
This document introduces convolutional neural networks (CNNs). It discusses how CNNs extract features using filters and pooling to build up representations of images while reducing the number of parameters. The key operations of CNNs including convolution, nonlinear activation, pooling and fully connected layers are explained. Examples of CNN applications are provided. The evolution of CNNs is then reviewed, from LeNet and AlexNet to VGGNet, GoogleNet, and improvements like ReLU, dropout, and batch normalization that helped CNNs train better and go deeper.
ResNet (short for Residual Network) is a deep neural network architecture that has achieved significant advancements in image recognition tasks. It was introduced by Kaiming He et al. in 2015.
The key innovation of ResNet is the use of residual connections, or skip connections, that enable the network to learn residual mappings instead of directly learning the desired underlying mappings. This addresses the problem of vanishing gradients that commonly occurs in very deep neural networks.
In a ResNet, the input data flows through a series of residual blocks. Each residual block consists of several convolutional layers followed by batch normalization and rectified linear unit (ReLU) activations. The original input to a residual block is passed through the block and added to the output of the block, creating a shortcut connection. This addition operation allows the network to learn residual mappings by computing the difference between the input and the output.
By using residual connections, the gradients can propagate more effectively through the network, enabling the training of deeper models. This enables the construction of extremely deep ResNet architectures with hundreds of layers, such as ResNet-101 or ResNet-152, while still maintaining good performance.
ResNet has become a widely adopted architecture in various computer vision tasks, including image classification, object detection, and image segmentation. Its ability to train very deep networks effectively has made it a fundamental building block in the field of deep learning.
Come puoi gestire i difetti? Se sei in una fabbrica, la produzione può produrre oggetti con difetti. Oppure i valori dei sensori possono dirti nel tempo che alcuni valori non sono "normali". Cosa puoi fare come sviluppatore (non come Data Scientist) con .NET o Azure per rilevare queste anomalie? Vediamo come in questa sessione.
Discovering Your AI Super Powers - Tips and Tricks to Jumpstart your AI ProjectsWee Hyong Tok
In this session, we will share about cutting-edge deep learning innovations, and present emerging trends in the AI community. This session is for data scientists, developers who have a keen interest in getting started in an AI project, and wants to learn the tools of the trade. We will draw on practical experiences from working on various AI projects, and share the key learning, and pitfalls
This document discusses using fully convolutional neural networks for defect inspection. It begins with an agenda that outlines image segmentation using FCNs and defect inspection. It then provides details on data preparation including labeling guidelines, data augmentation, and model setup using techniques like deconvolution layers and the U-Net architecture. Metrics for evaluating the model like Dice score and IoU are also covered. The document concludes with best practices for successful deep learning projects focusing on aspects like having a large reusable dataset, feasibility of the problem, potential payoff, and fault tolerance.
https://telecombcn-dl.github.io/2019-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
Le Song, Assistant Professor, College of Computing, Georgia Institute of Tech...MLconf
Understanding Deep Learning for Big Data: The complexity and scale of big data impose tremendous challenges for their analysis. Yet, big data also offer us great opportunities. Some nonlinear phenomena, features or relations, which are not clear or cannot be inferred reliably from small and medium data, now become clear and can be learned robustly from big data. Typically, the form of the nonlinearity is unknown to us, and needs to be learned from data as well. Being able to harness the nonlinear structures from big data could allow us to tackle problems which are impossible before or obtain results which are far better than previous state-of-the-arts.
Nowadays, deep neural networks are the methods of choice when it comes to large scale nonlinear learning problems. What makes deep neural networks work? Is there any general principle for tackling high dimensional nonlinear problems which we can learn from deep neural works? Can we design competitive or better alternatives based on such knowledge? To make progress in these questions, my machine learning group performed both theoretical and experimental analysis on existing and new deep learning architectures, and investigate three crucial aspects on the usefulness of the fully connected layers, the advantage of the feature learning process, and the importance of the compositional structures. Our results point to some promising directions for future research, and provide guideline for building new deep learning models.
How can you handle defects? If you are in a factory, production can produce objects with defects. Or values from sensors can tell you over time that some values are not "normal". What can you do as a developer (not a Data Scientist) with .NET o Azure to detect these anomalies? Let's see how in this session.
The document discusses word embedding techniques used to represent words as vectors. It describes Word2Vec as a popular word embedding model that uses either the Continuous Bag of Words (CBOW) or Skip-gram architecture. CBOW predicts a target word based on surrounding context words, while Skip-gram predicts surrounding words given a target word. These models represent words as dense vectors that encode semantic and syntactic properties, allowing operations like word analogy questions.
Ring loss: Convex Feature Normalization for Face Recognition郁凱 黃
Ring loss is a feature normalization approach for deep networks that augments standard loss functions like softmax. It encourages the norm of sample features to be a learned parameter R rather than enforcing hard normalization. This soft normalization helps address issues with imbalanced classification margins and disconnect between training and testing metrics due to variation in feature norms. Experiments on large face recognition datasets show Ring loss improves performance compared to softmax, especially for low resolution images where feature norms are typically lower. It achieves state-of-the-art results on benchmarks like LFW, IJB-A, MegaFace, and CFP.
- Support vector machines (SVMs) find a linear separator between classes that maximizes the margin between the separator and the nearest data points of each class. This maximum-margin separator generalizes better than other possible separators.
- SVMs can learn nonlinear decision boundaries by mapping data into a high-dimensional feature space and finding a linear separator in that space, which corresponds to a nonlinear separator in the original input space.
- The "kernel trick" allows SVMs to efficiently compute scalar products between points in the high-dimensional feature space without explicitly performing the mapping, making SVMs practical even with huge numbers of features.
- Support vector machines (SVMs) are a machine learning method for classification and regression. They find the optimal separating hyperplane between classes that maximizes the margin between the plane and the closest data points.
- SVMs use a "kernel trick" to efficiently perform computations in high-dimensional feature spaces without explicitly computing the coordinates of data in that space. Common kernels include polynomial and Gaussian radial basis function kernels.
- To classify new examples, SVMs use a decision function that depends on a subset of training samples called support vectors. The model is defined by these support vectors and weights learned during training.
- Support vector machines (SVMs) find a linear separator between classes that maximizes the margin between the separator and the closest data points. This maximum margin separator generalizes better than other separators.
- SVMs can handle non-linear separations by projecting data into a higher-dimensional feature space and finding a linear separator there. The kernel trick allows efficient computation without explicitly using the high-dimensional feature space.
- SVMs solve a convex optimization problem to find the maximum margin separator. Only a subset of data points called support vectors are used to define the separator and classify new data.
This document discusses Command Query Responsibility Segregation (CQRS). It begins by explaining that CQRS splits commands and queries into separate paths at an architectural level. This aligns with how people think about tasks (commands) and requesting information (queries) differently. The document then discusses how CQRS provides an eventually consistent architecture based on CAP theorem, similar to BASE. It explains that CQRS reads data from a separate read model than where commands write to the domain model. In summary, CQRS separates commands from queries in a way that matches how people think, and provides an eventually consistent architecture.
PR-297: Training data-efficient image transformers & distillation through att...Jinwon Lee
안녕하세요 TensorFlow Korea 논문 읽기 모임 PR-12의 297번째 리뷰입니다
어느덧 PR-12 시즌 3의 끝까지 논문 3편밖에 남지 않았네요.
시즌 3가 끝나면 바로 시즌 4의 새 멤버 모집이 시작될 예정입니다. 많은 관심과 지원 부탁드립니다~~
(멤버 모집 공지는 Facebook TensorFlow Korea 그룹에 올라올 예정입니다)
오늘 제가 리뷰한 논문은 Facebook의 Training data-efficient image transformers & distillation through attention 입니다.
Google에서 나왔던 ViT논문 이후에 convolution을 전혀 사용하지 않고 오직 attention만을 이용한 computer vision algorithm에 어느때보다 관심이 높아지고 있는데요
이 논문에서 제안한 DeiT 모델은 ViT와 같은 architecture를 사용하면서 ViT가 ImageNet data만으로는 성능이 잘 안나왔던 것에 비해서
Training 방법 개선과 새로운 Knowledge Distillation 방법을 사용하여 mageNet data 만으로 EfficientNet보다 뛰어난 성능을 보여주는 결과를 얻었습니다.
정말 CNN은 이제 서서히 사라지게 되는 것일까요? Attention이 computer vision도 정복하게 될 것인지....
개인적으로는 당분간은 attention 기반의 CV 논문이 쏟아질 거라고 확신하고, 또 여기에서 놀라운 일들이 일어날 수 있을 거라고 생각하고 있습니다
CNN은 10년간 많은 연구를 통해서 발전해왔지만, transformer는 이제 CV에 적용된 지 얼마 안된 시점이라서 더 기대가 크구요,
attention이 inductive bias가 가장 적은 형태의 모델이기 때문에 더 놀라운 이들을 만들 수 있을거라고 생각합니다
얼마 전에 나온 open AI의 DALL-E도 그 대표적인 예라고 할 수 있을 것 같습니다. Transformer의 또하나의 transformation이 궁금하신 분들은 아래 영상을 참고해주세요
영상링크: https://youtu.be/DjEvzeiWBTo
논문링크: https://arxiv.org/abs/2012.12877
Similar to Paper overview: "Deep Residual Learning for Image Recognition" (20)
Understanding Information Processing in Human Brain by Interpreting Machine L...Ilya Kuzovkin
The thesis explores the role machine learning methods play in creating intuitive computational models of neural processing. Combined with interpretability techniques, machine learning could replace human modeler and shift the focus of human effort to extracting the knowledge from the ready-made models and articulating that knowledge into intuitive descriptions of reality. This perspective makes the case in favor of the larger role that exploratory and data-driven approach to computational neuroscience could play while coexisting alongside the traditional hypothesis-driven approach.
We exemplify the proposed approach in the context of the knowledge representation taxonomy with three research projects that employ interpretability techniques on top of machine learning methods at three different levels of neural organization. The first study (Chapter 3) explores feature importance analysis of a random forest decoder trained on intracerebral recordings from 100 human subjects to identify spectrotemporal signatures that characterize local neural activity during the task of visual categorization. The second study (Chapter 4) employs representation similarity analysis to compare the neural responses of the areas along the ventral stream with the activations of the layers of a deep convolutional neural network. The third study (Chapter 5) proposes a method that allows test subjects to visually explore the state representation of their neural signal in real time. This is achieved by using a topology-preserving dimensionality reduction technique that allows to transform the neural data from the multidimensional representation used by the computer into a two-dimensional representation a human can grasp.
The approach, the taxonomy, and the examples, present a strong case for the applicability of machine learning methods to automatic knowledge discovery in neuroscience.
The Brain and the Modern AI: Drastic Differences and Curious SimilaritiesIlya Kuzovkin
In this presentation we user Marr's three levels of analysis to approach the question of how similar or different are the brain and the machines. By approaching this discussion in a more structured manner we find that the answer is different depending on the level of abstraction we are operating at.
Introductory talk about how to think about various deep learning architectures and their suitability to different types of data. Presented at Data Science Meetup @ University of Tartu
Intuitive Intro to Gödel's Incompleteness TheoremIlya Kuzovkin
Intuitive introduction into meaning of Gödel's incompleteness theorem. Slides are prepared for "Philosophy of Artificial Intelligence" class at University of Tartu
Introduction to Machine Learning @ Mooncascade ML CampIlya Kuzovkin
The document discusses machine learning concepts including representing data in a machine-readable format through feature extraction, choosing an algorithm such as decision trees, evaluating accuracy using a confusion matrix, and ensuring a model generalizes well to new data. Key steps are extracting features from raw data to create an instance, selecting an algorithm like decision trees to learn patterns in the data, and evaluating accuracy on new data while avoiding overfitting or imbalanced training data.
Mastering the game of Go with deep neural networks and tree search (article o...Ilya Kuzovkin
"This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away." In this presentation I go through the pipeline following the steps the algorithm does to understand the process.
Deep Learning: Theory, History, State of the Art & Practical ToolsIlya Kuzovkin
The document provides an overview of the history and development of deep learning, beginning with early work on artificial neurons in the 1940s-50s and the perceptron in the 1950s. It describes how backpropagation and use of the sigmoid function in the 1980s enabled training of multi-layer neural networks, but that the deep learning revolution did not occur until recently due to limitations in data, computing power, and other factors at the time. The remainder of the document focuses on explaining how backpropagation works to train neural networks by calculating error in the forward pass and using it to update weights in the backward pass through gradient descent.
Article overview: Unsupervised Learning of Visual Structure Using Predictive ...Ilya Kuzovkin
This set of slides goes over the recent article that tries to tie together the idea of predictive coding and deep learning. The main point of the article is that a generative system trained on sequential data to predict the future samples learns more "useful" representation than the usual autoencoder. The result resonates with the fact that our brain is probably using predictive mechanisms.
Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of...Ilya Kuzovkin
The article presents the comparison of the complexity of the representation of visual features in the deep convolutional neural network and in our brain. DNN activity layer-by-layer is used to predict voxel activations and it is shown that lower layers of DNN are better at predicting V1,V2 and that higher layers of DNN are better in predicting activity in LO and higher areas of ventral stream. The result effectively demonstrates that layer-by-layer complexity of visual features we see in DNN is also present in the visual cortex.
NIPS2014 Article Overview: Do Deep Nets Really Need to be Deep?Ilya Kuzovkin
Year 2014 passed under the huge Deep Learning sign. At the last seminar of computational neuroscience I've presented a very recent article, which looks into the question of whether deepness in a requirements for approximating a function, or, under the right condition, shallow nets are capable of performing equally well?
The Neural Turing Machine is a neural network architecture inspired by biological working memory and digital computers that is differentiable end-to-end and can be trained with gradient descent. Experiments showed it is capable of learning simple algorithms from example data and generalizing well outside its training data.
Introductory lecture on neuroimaging techniques: intracortical, fMRI, EEG. Tends to explain the ideas of the technologies on a good level of intuition. Presented at AACIMP'14 (http://summerschool.ssa.org.ua/program/42-program/ns-2014/442-machine-learning-on-neuroimaging-data)
Article Overview "Reach and grasp by people with tetraplegia using a neurally...Ilya Kuzovkin
This presentation is article overview given at the Computational Neuroscience seminar in the University of Tartu. In my opinion at the moment this is the most prominent BCI system out there.
The document provides an introduction to GPU computing. It begins with a simple OpenGL program to demonstrate parallelism on the GPU. It then discusses shader programs and lighting effects. Next, it covers using the OpenGL pipeline and GLSL to perform computations by rendering to a framebuffer object. The document introduces compute shaders and compares CUDA and OpenCL programming models. It provides a code example and discusses resources for learning more about GPU computing.
Ilya Kuzovkin - Adaptive Interactive Learning for Brain-Computer InterfacesIlya Kuzovkin
This is a brief description of the approach and algorithm I proposed in my master's thesis. The core algorithm keeps track of both user's and computer's progress and gives them valuable feedback.
Immersive Learning That Works: Research Grounding and Paths ForwardLeonel Morgado
We will metaverse into the essence of immersive learning, into its three dimensions and conceptual models. This approach encompasses elements from teaching methodologies to social involvement, through organizational concerns and technologies. Challenging the perception of learning as knowledge transfer, we introduce a 'Uses, Practices & Strategies' model operationalized by the 'Immersive Learning Brain' and ‘Immersion Cube’ frameworks. This approach offers a comprehensive guide through the intricacies of immersive educational experiences and spotlighting research frontiers, along the immersion dimensions of system, narrative, and agency. Our discourse extends to stakeholders beyond the academic sphere, addressing the interests of technologists, instructional designers, and policymakers. We span various contexts, from formal education to organizational transformation to the new horizon of an AI-pervasive society. This keynote aims to unite the iLRN community in a collaborative journey towards a future where immersive learning research and practice coalesce, paving the way for innovative educational research and practice landscapes.
ESR spectroscopy in liquid food and beverages.pptxPRIYANKA PATEL
With increasing population, people need to rely on packaged food stuffs. Packaging of food materials requires the preservation of food. There are various methods for the treatment of food to preserve them and irradiation treatment of food is one of them. It is the most common and the most harmless method for the food preservation as it does not alter the necessary micronutrients of food materials. Although irradiated food doesn’t cause any harm to the human health but still the quality assessment of food is required to provide consumers with necessary information about the food. ESR spectroscopy is the most sophisticated way to investigate the quality of the food and the free radicals induced during the processing of the food. ESR spin trapping technique is useful for the detection of highly unstable radicals in the food. The antioxidant capability of liquid food and beverages in mainly performed by spin trapping technique.
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...Sérgio Sacani
Context. With a mass exceeding several 104 M⊙ and a rich and dense population of massive stars, supermassive young star clusters
represent the most massive star-forming environment that is dominated by the feedback from massive stars and gravitational interactions
among stars.
Aims. In this paper we present the Extended Westerlund 1 and 2 Open Clusters Survey (EWOCS) project, which aims to investigate
the influence of the starburst environment on the formation of stars and planets, and on the evolution of both low and high mass stars.
The primary targets of this project are Westerlund 1 and 2, the closest supermassive star clusters to the Sun.
Methods. The project is based primarily on recent observations conducted with the Chandra and JWST observatories. Specifically,
the Chandra survey of Westerlund 1 consists of 36 new ACIS-I observations, nearly co-pointed, for a total exposure time of 1 Msec.
Additionally, we included 8 archival Chandra/ACIS-S observations. This paper presents the resulting catalog of X-ray sources within
and around Westerlund 1. Sources were detected by combining various existing methods, and photon extraction and source validation
were carried out using the ACIS-Extract software.
Results. The EWOCS X-ray catalog comprises 5963 validated sources out of the 9420 initially provided to ACIS-Extract, reaching a
photon flux threshold of approximately 2 × 10−8 photons cm−2
s
−1
. The X-ray sources exhibit a highly concentrated spatial distribution,
with 1075 sources located within the central 1 arcmin. We have successfully detected X-ray emissions from 126 out of the 166 known
massive stars of the cluster, and we have collected over 71 000 photons from the magnetar CXO J164710.20-455217.
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...Advanced-Concepts-Team
Presentation in the Science Coffee of the Advanced Concepts Team of the European Space Agency on the 07.06.2024.
Speaker: Diego Blas (IFAE/ICREA)
Title: Gravitational wave detection with orbital motion of Moon and artificial
Abstract:
In this talk I will describe some recent ideas to find gravitational waves from supermassive black holes or of primordial origin by studying their secular effect on the orbital motion of the Moon or satellites that are laser ranged.
PPT on Direct Seeded Rice presented at the three-day 'Training and Validation Workshop on Modules of Climate Smart Agriculture (CSA) Technologies in South Asia' workshop on April 22, 2024.
The debris of the ‘last major merger’ is dynamically youngSérgio Sacani
The Milky Way’s (MW) inner stellar halo contains an [Fe/H]-rich component with highly eccentric orbits, often referred to as the
‘last major merger.’ Hypotheses for the origin of this component include Gaia-Sausage/Enceladus (GSE), where the progenitor
collided with the MW proto-disc 8–11 Gyr ago, and the Virgo Radial Merger (VRM), where the progenitor collided with the
MW disc within the last 3 Gyr. These two scenarios make different predictions about observable structure in local phase space,
because the morphology of debris depends on how long it has had to phase mix. The recently identified phase-space folds in Gaia
DR3 have positive caustic velocities, making them fundamentally different than the phase-mixed chevrons found in simulations
at late times. Roughly 20 per cent of the stars in the prograde local stellar halo are associated with the observed caustics. Based
on a simple phase-mixing model, the observed number of caustics are consistent with a merger that occurred 1–2 Gyr ago.
We also compare the observed phase-space distribution to FIRE-2 Latte simulations of GSE-like mergers, using a quantitative
measurement of phase mixing (2D causticality). The observed local phase-space distribution best matches the simulated data
1–2 Gyr after collision, and certainly not later than 3 Gyr. This is further evidence that the progenitor of the ‘last major merger’
did not collide with the MW proto-disc at early times, as is thought for the GSE, but instead collided with the MW disc within
the last few Gyr, consistent with the body of work surrounding the VRM.
Sharlene Leurig - Enabling Onsite Water Use with Net Zero Water
Paper overview: "Deep Residual Learning for Image Recognition"
1. Article overview by
Ilya Kuzovkin
K. He, X. Zhang, S. Ren and J. Sun
Microsoft Research
Computational Neuroscience Seminar
University of Tartu
2016
Deep Residual Learning for
Image Recognition ILSVRC 2015
MS COCO 2015
WINNER
22. Add explicit identity connections and “solvers
may simply drive the weights of the multiple
nonlinear layers toward zero”
is the true function we
want to learn
23. Add explicit identity connections and “solvers
may simply drive the weights of the multiple
nonlinear layers toward zero”
is the true function we
want to learn
Let’s pretend we want to learn
instead.
24. Add explicit identity connections and “solvers
may simply drive the weights of the multiple
nonlinear layers toward zero”
is the true function we
want to learn
Let’s pretend we want to learn
instead.
The original function is then
31. • Lots of convolutional 3x3 layers
• VGG complexity is 19.6 billion FLOPs
34-layer-ResNet is 3.6 bln. FLOPs
32. • Lots of convolutional 3x3 layers
• VGG complexity is 19.6 billion FLOPs
34-layer-ResNet is 3.6 bln. FLOPs
!
• Batch normalization
• SGD with batch size 256
• (up to) 600,000 iterations
• LR 0.1 (divided by 10 when error plateaus)
• Momentum 0.9
• No dropout
• Weight decay 0.0001
33. • Lots of convolutional 3x3 layers
• VGG complexity is 19.6 billion FLOPs
34-layer-ResNet is 3.6 bln. FLOPs
!
• Batch normalization
• SGD with batch size 256
• (up to) 600,000 iterations
• LR 0.1 (divided by 10 when error plateaus)
• Momentum 0.9
• No dropout
• Weight decay 0.0001
!
• 1.28 million training images
• 50,000 validation
• 100,000 test
34. • 34-layer ResNet has lower training error.
This indicates that the degradation
problem is well addressed and we
manage to obtain accuracy gains from
increased depth.
35. • 34-layer ResNet has lower training error.
This indicates that the degradation
problem is well addressed and we
manage to obtain accuracy gains from
increased depth.
!
• 34-layer-ResNet reduces the top-1 error
by 3.5%
36. • 34-layer ResNet has lower training error.
This indicates that the degradation
problem is well addressed and we
manage to obtain accuracy gains from
increased depth.
!
• 34-layer-ResNet reduces the top-1 error
by 3.5%
!
• 18-layer ResNet converges faster and
thus ResNet eases the optimization by
providing faster convergence at the
early stage.