The document discusses exploring randomly wired neural networks for image recognition. It introduces randomly wired neural networks as a new approach to neural architecture search. Random network generators are used to stochastically sample network topologies. Experiments show that randomly wired networks can achieve competitive accuracy to hand-designed and NAS networks on ImageNet classification, using fewer resources than typical NAS. The authors hope further exploring network generator designs will yield more powerful network topologies.
PR-155: Exploring Randomly Wired Neural Networks for Image RecognitionJinwon Lee
TensorFlow-KR 논문읽기모임 PR12 155번째 논문 review 입니다.
이번에는 Facebook AI Research에서 최근에 나온(4/2) Exploring Randomly Wired Neural Networks for Image Recognition을 review해 보았습니다. random하게 generation된 network이 그동안 사람들이 온갖 노력을 들여서 만든 network 이상의 성능을 나타낸다는 결과로 많은 사람들에게 충격을 준 논문인데요, 자세한 내용은 자료와 영상을 참고해주세요
논문링크: https://arxiv.org/abs/1904.01569
영상링크: https://youtu.be/NrmLteQ5BC4
PR-183: MixNet: Mixed Depthwise Convolutional KernelsJinwon Lee
TensorFlow-KR 논문읽기모임 PR12(12PR) 183번째 논문 review입니다.
이번에 살펴볼 논문은 Google Brain에서 발표한 MixNet입니다. Efficiency를 추구하는 CNN에서 depthwise convolution이 많이 사용되는데, 이 때 depthwise convolution filter의 size를 다양하게 해서 성능도 높이고 efficiency도 높이는 방법을 제안한 논문입니다. 자세한 내용은 영상을 참고해주세요
논문링크 : https://arxiv.org/abs/1907.09595
발표영상 : https://youtu.be/252YxqpHzsg
PR-169: EfficientNet: Rethinking Model Scaling for Convolutional Neural NetworksJinwon Lee
TensorFlow-KR 논문읽기모임 PR12 169번째 논문 review입니다.
이번에 살펴본 논문은 Google에서 발표한 EfficientNet입니다. efficient neural network은 보통 mobile과 같은 제한된 computing power를 가진 edge device를 위한 작은 network 위주로 연구되어왔는데, 이 논문은 성능을 높이기 위해서 일반적으로 network를 점점 더 키워나가는 경우가 많은데, 이 때 어떻게 하면 더 효율적인 방법으로 network을 키울 수 있을지에 대해서 연구한 논문입니다. 자세한 내용은 영상을 참고해주세요
논문링크: https://arxiv.org/abs/1905.11946
영상링크: https://youtu.be/Vhz0quyvR7I
201907 AutoML and Neural Architecture SearchDaeJin Kim
Brief introduction of NAS
Review of EfficientNet (Google Brain), RandWire (FAIR) papers
NAS flow slide from KihoSuh's slideshare (https://www.slideshare.net/KihoSuh/neural-architecture-search-with-reinforcement-learning-76883153)
[References]
[1] EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (https://arxiv.org/abs/1905.11946)
[2] Exploring Randomly Wired Neural Networks for Image Recognition (https://arxiv.org/abs/1904.01569)
PR-217: EfficientDet: Scalable and Efficient Object DetectionJinwon Lee
TensorFlow Korea 논문읽기모임 PR12 217번째 논문 review입니다
이번 논문은 GoogleBrain에서 쓴 EfficientDet입니다. EfficientNet의 후속작으로 accuracy와 efficiency를 둘 다 잡기 위한 object detection 방법을 제안한 논문입니다. 이를 위하여 weighted bidirectional feature pyramid network(BiFPN)과 EfficientNet과 유사한 방법의 detection용 compound scaling 방법을 제안하고 있는데요, 자세한 내용은 영상을 참고해주세요
논문링크: https://arxiv.org/abs/1911.09070
영상링크: https://youtu.be/11jDC8uZL0E
PR-197: One ticket to win them all: generalizing lottery ticket initializatio...Jinwon Lee
TensorFlow Korea 논문읽기모임 PR12 197번째 논문 review입니다
(2기 목표 200편까지 이제 3편이 남았습니다)
이번에 제가 발표한 논문은 FAIR(Facebook AI Research)에서 나온 One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers 입니다
한 장의 ticket으로 모든 복권에서 1등을 할 수 있다면 얼마나 좋을까요?
일반적인 network pruning 방법은 pruning 하기 이전에 학습된 network weight를 그대로 사용하면서 fine tuning하는 방법을 사용해왔습니다
pruning한 이후에 network에 weight를 random intialization한 후 학습하면 성능이 잘 나오지 않는 문제가 있었는데요
작년 MIT에서 나온 Lottery ticket hypothesis라는 논문에서는 이렇게 pruning된 이후의 network를 어떻게 random intialization하면 높은 성능을 낼 수 있는지
이 intialization 방법을 공개하며 lottery ticket의 winning ticket이라고 이름붙였습니다.
그런데 이 winning ticket이 혹시 다른 dataset이나 다른 optimizer를 사용하는 경우에도 잘 동작할 수 있을까요?
예를 들어 CIFAR10에서 찾은 winning ticket이 ImageNet에서도 winning ticket의 성능을 나타낼 수 있을까요?
이 논문은 이러한 질문에 대한 답을 실험을 통해서 확인하였고, initialization에 대한 여러가지 insight를 담고 있습니다.
자세한 내용은 발표 영상을 참고해주세요~!
영상링크: https://youtu.be/YmTNpF2OOjA
발표자료링크: https://www.slideshare.net/JinwonLee9/pr197-one-ticket-to-win-them-all-generalizing-lottery-ticket-initializations-across-datasets-and-optimizers
논문링크: https://arxiv.org/abs/1906.02773
PR-155: Exploring Randomly Wired Neural Networks for Image RecognitionJinwon Lee
TensorFlow-KR 논문읽기모임 PR12 155번째 논문 review 입니다.
이번에는 Facebook AI Research에서 최근에 나온(4/2) Exploring Randomly Wired Neural Networks for Image Recognition을 review해 보았습니다. random하게 generation된 network이 그동안 사람들이 온갖 노력을 들여서 만든 network 이상의 성능을 나타낸다는 결과로 많은 사람들에게 충격을 준 논문인데요, 자세한 내용은 자료와 영상을 참고해주세요
논문링크: https://arxiv.org/abs/1904.01569
영상링크: https://youtu.be/NrmLteQ5BC4
PR-183: MixNet: Mixed Depthwise Convolutional KernelsJinwon Lee
TensorFlow-KR 논문읽기모임 PR12(12PR) 183번째 논문 review입니다.
이번에 살펴볼 논문은 Google Brain에서 발표한 MixNet입니다. Efficiency를 추구하는 CNN에서 depthwise convolution이 많이 사용되는데, 이 때 depthwise convolution filter의 size를 다양하게 해서 성능도 높이고 efficiency도 높이는 방법을 제안한 논문입니다. 자세한 내용은 영상을 참고해주세요
논문링크 : https://arxiv.org/abs/1907.09595
발표영상 : https://youtu.be/252YxqpHzsg
PR-169: EfficientNet: Rethinking Model Scaling for Convolutional Neural NetworksJinwon Lee
TensorFlow-KR 논문읽기모임 PR12 169번째 논문 review입니다.
이번에 살펴본 논문은 Google에서 발표한 EfficientNet입니다. efficient neural network은 보통 mobile과 같은 제한된 computing power를 가진 edge device를 위한 작은 network 위주로 연구되어왔는데, 이 논문은 성능을 높이기 위해서 일반적으로 network를 점점 더 키워나가는 경우가 많은데, 이 때 어떻게 하면 더 효율적인 방법으로 network을 키울 수 있을지에 대해서 연구한 논문입니다. 자세한 내용은 영상을 참고해주세요
논문링크: https://arxiv.org/abs/1905.11946
영상링크: https://youtu.be/Vhz0quyvR7I
201907 AutoML and Neural Architecture SearchDaeJin Kim
Brief introduction of NAS
Review of EfficientNet (Google Brain), RandWire (FAIR) papers
NAS flow slide from KihoSuh's slideshare (https://www.slideshare.net/KihoSuh/neural-architecture-search-with-reinforcement-learning-76883153)
[References]
[1] EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (https://arxiv.org/abs/1905.11946)
[2] Exploring Randomly Wired Neural Networks for Image Recognition (https://arxiv.org/abs/1904.01569)
PR-217: EfficientDet: Scalable and Efficient Object DetectionJinwon Lee
TensorFlow Korea 논문읽기모임 PR12 217번째 논문 review입니다
이번 논문은 GoogleBrain에서 쓴 EfficientDet입니다. EfficientNet의 후속작으로 accuracy와 efficiency를 둘 다 잡기 위한 object detection 방법을 제안한 논문입니다. 이를 위하여 weighted bidirectional feature pyramid network(BiFPN)과 EfficientNet과 유사한 방법의 detection용 compound scaling 방법을 제안하고 있는데요, 자세한 내용은 영상을 참고해주세요
논문링크: https://arxiv.org/abs/1911.09070
영상링크: https://youtu.be/11jDC8uZL0E
PR-197: One ticket to win them all: generalizing lottery ticket initializatio...Jinwon Lee
TensorFlow Korea 논문읽기모임 PR12 197번째 논문 review입니다
(2기 목표 200편까지 이제 3편이 남았습니다)
이번에 제가 발표한 논문은 FAIR(Facebook AI Research)에서 나온 One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers 입니다
한 장의 ticket으로 모든 복권에서 1등을 할 수 있다면 얼마나 좋을까요?
일반적인 network pruning 방법은 pruning 하기 이전에 학습된 network weight를 그대로 사용하면서 fine tuning하는 방법을 사용해왔습니다
pruning한 이후에 network에 weight를 random intialization한 후 학습하면 성능이 잘 나오지 않는 문제가 있었는데요
작년 MIT에서 나온 Lottery ticket hypothesis라는 논문에서는 이렇게 pruning된 이후의 network를 어떻게 random intialization하면 높은 성능을 낼 수 있는지
이 intialization 방법을 공개하며 lottery ticket의 winning ticket이라고 이름붙였습니다.
그런데 이 winning ticket이 혹시 다른 dataset이나 다른 optimizer를 사용하는 경우에도 잘 동작할 수 있을까요?
예를 들어 CIFAR10에서 찾은 winning ticket이 ImageNet에서도 winning ticket의 성능을 나타낼 수 있을까요?
이 논문은 이러한 질문에 대한 답을 실험을 통해서 확인하였고, initialization에 대한 여러가지 insight를 담고 있습니다.
자세한 내용은 발표 영상을 참고해주세요~!
영상링크: https://youtu.be/YmTNpF2OOjA
발표자료링크: https://www.slideshare.net/JinwonLee9/pr197-one-ticket-to-win-them-all-generalizing-lottery-ticket-initializations-across-datasets-and-optimizers
논문링크: https://arxiv.org/abs/1906.02773
PR-144: SqueezeNext: Hardware-Aware Neural Network DesignJinwon Lee
Tensorfkow-KR 논문읽기모임 PR12 144번째 논문 review입니다.
이번에는 Efficient CNN의 대표 중 하나인 SqueezeNext를 review해보았습니다. SqueezeNext의 전신인 SqueezeNet도 같이 review하였고, CNN을 평가하는 metric에 대한 논문인 NetScore에서 SqueezeNext가 1등을 하여 NetScore도 같이 review하였습니다.
논문링크:
SqueezeNext - https://arxiv.org/abs/1803.10615
SqueezeNet - https://arxiv.org/abs/1602.07360
NetScore - https://arxiv.org/abs/1806.05512
영상링크: https://youtu.be/WReWeADJ3Pw
PR095: Modularity Matters: Learning Invariant Relational Reasoning TasksJinwon Lee
Tensorflow-KR 논문읽기모임 95번째 발표영상입니다
Modularity Matters라는 제목으로 visual relational reasoning 문제를 풀 수 있는 방법을 제시한 논문입니다. 기존 CNN들이 이런 문제이 취약함을 보여주고 이를 해결하기 위한 방법을 제시합니다. 관심있는 주제이기도 하고 Bengio 교수님 팀에서 쓴 논문이라서 review 해보았습니다
발표영상: https://youtu.be/dAGI3mlOmfw
논문링크: https://arxiv.org/abs/1806.06765
TensorFlow Korea 논문읽기모임 PR12 243째 논문 review입니다
이번 논문은 RegNet으로 알려진 Facebook AI Research의 Designing Network Design Spaces 입니다.
CNN을 디자인할 때, bottleneck layer는 정말 좋을까요? layer 수는 많을 수록 높은 성능을 낼까요? activation map의 width, height를 절반으로 줄일 때(stride 2 혹은 pooling), channel을 2배로 늘려주는데 이게 최선일까요? 혹시 bottleneck layer가 없는 게 더 좋지는 않은지, 최고 성능을 내는 layer 수에 magic number가 있는 건 아닐지, activation이 절반으로 줄어들 때 channel을 2배가 아니라 3배로 늘리는 게 더 좋은건 아닌지?
이 논문에서는 하나의 neural network을 잘 design하는 것이 아니라 Auto ML과 같은 기술로 좋은 neural network을 찾을 수 있는 즉 좋은 neural network들이 살고 있는 좋은 design space를 design하는 방법에 대해서 얘기하고 있습니다. constraint이 거의 없는 design space에서 human-in-the-loop을 통해 좋은 design space로 그 공간을 좁혀나가는 방법을 제안하였는데요, EfficientNet보다 더 좋은 성능을 보여주는 RegNet은 어떤 design space에서 탄생하였는지 그리고 그 과정에서 우리가 당연하게 여기고 있었던 design choice들이 잘못된 부분은 없었는지 아래 동영상에서 확인하실 수 있습니다~
영상링크: https://youtu.be/bnbKQRae_u4
논문링크: https://arxiv.org/abs/2003.13678
PR-231: A Simple Framework for Contrastive Learning of Visual RepresentationsJinwon Lee
TensorFlow Korea 논문읽기모임 PR12 231번째 논문 review 입니다
이번 논문은 Google Brain에서 나온 A Simple Framework for Contrastive Learning of Visual Representations입니다. Geoffrey Hinton님이 마지막 저자이시기도 해서 최근에 더 주목을 받고 있는 논문입니다.
이 논문은 최근에 굉장히 핫한 topic인 contrastive learning을 이용한 self-supervised learning쪽 논문으로 supervised learning으로 학습한 ResNet50와 동일한 성능을 얻을 수 있는 unsupervised pre-trainig 방법을 제안하였습니다. Data augmentation, Non-linear projection head, large batch size, longer training, NTXent loss 등을 활용하여 훌륭한 representation learning이 가능함을 보여주었고, semi-supervised learning이나 transfer learning에서도 매우 뛰어난 결과를 보여주었습니다. 자세한 내용은 영상을 참고해주세요
논문링크: https://arxiv.org/abs/2002.05709
영상링크: https://youtu.be/FWhM3juUM6s
PR-284: End-to-End Object Detection with Transformers(DETR)Jinwon Lee
TensorFlow Korea 논문읽기모임 PR12 284번째 논문 review입니다.
이번 논문은 Facebook에서 나온 DETR(DEtection with TRansformer) 입니다.
arxiv-sanity에 top recent/last year에서 가장 상위에 자리하고 있는 논문이기도 합니다(http://www.arxiv-sanity.com/top?timefilter=year&vfilter=all)
최근에 ICLR 2021에 submit된 ViT로 인해서 이제 Transformer가 CNN을 대체하는 것 아닌가 하는 얘기들이 많이 나오고 있는데요, 올 해 ECCV에 발표된 논문이고 feature extraction 부분은 CNN을 사용하긴 했지만 transformer를 활용하여 효과적으로 Object Detection을 수행하는 방법을 제안한 중요한 논문이라고 생각합니다. 이 논문에서는 detection 문제에서 anchor box나 NMS(Non Maximum Supression)와 같은 heuristic 하고 미분 불가능한 방법들이 많이 사용되고, 이로 인해서 유독 object detection 문제는 딥러닝의 철학인 end-to-end 방식으로 해결되지 못하고 있음을 지적하고 있습니다. 그 해결책으로 bounding box를 예측하는 문제를 set prediction problem(중복을 허용하지 않고, 순서에 무관함)으로 보고 transformer를 활용한 end-to-end 방식의 알고리즘을 제안하였습니다. anchor box도 필요없고 NMS도 필요없는 DETR 알고리즘의 자세한 내용이 알고싶으시면 영상을 참고해주세요!
영상링크: https://youtu.be/lXpBcW_I54U
논문링크: https://arxiv.org/abs/2005.12872
PR-270: PP-YOLO: An Effective and Efficient Implementation of Object DetectorJinwon Lee
TensorFlow Korea 논문읽기모임 PR12 270번째 논문 review입니다.
이번 논문은 Baidu에서 나온 PP-YOLO: An Effective and Efficient Implementation of Object Detector입니다. YOLOv3에 다양한 방법을 적용하여 매우 높은 성능과 함께 매우 빠른 속도 두마리 토끼를 다 잡아버린(?) 그런 논문입니다. 논문에서 사용한 다양한 trick들에 대해서 좀 더 깊이있게 살펴보았습니다. Object detection에 사용된 기법 들 중에 Deformable convolution, Exponential Moving Average, DropBlock, IoU aware prediction, Grid sensitivity elimination, MatrixNMS, CoordConv, 등의 방법에 관심이 있으시거나 알고 싶으신 분들은 영상과 발표자료를 참고하시면 좋을 것 같습니다!
논문링크: https://arxiv.org/abs/2007.12099
영상링크: https://youtu.be/7v34cCE5H4k
In machine learning, a convolutional neural network is a class of deep, feed-forward artificial neural networks that have successfully been applied fpr analyzing visual imagery.
[2020 CVPR Efficient DET 논문리뷰]
안녕하세요 딥러닝 논문읽기 모임입니다.
오늘 소개드릴 논문은 2020 CVPR에서 발표된, Efficient Net 저자가 발표한 'Efficient DET'입니다. 제목에서 유추 가능하듯 Backbone을 Efficient Net으로 사용하여 Object Detection Task에 적용 했다는 점을 유추할 수 있습니다.
해당 논문은 위 사실을 제외하고, 조금 특별한 Feature Pyramid Network를 적용하여 더욱 성능적으로 향상을 시켰는대요.
이러한 내용을 바탕으로 아직까지도 paperswithcode에 상위권에 랭크되어 있는 논문 입니다.
오늘 논문 이미지처리팀 이찬혁님이 자세하고 디테일한 리뷰를 도와주셨습니다.
많은 관심 미리 감사드립니다!
https://youtu.be/Mq4aqDgZ2bI
Explores the type of structure learned by Convolutional Neural Networks, the applications where they're most valuable and a number of appropriate mental models for understanding deep learning.
In this presentation we discuss the convolution operation, the architecture of a convolution neural network, different layers such as pooling etc. This presentation draws heavily from A Karpathy's Stanford Course CS 231n
A Framework for Scene Recognition Using Convolutional Neural Network as Featu...Tahmid Abtahi
Scene recognition is one of the hallmark tasks of computer vision, allowing definition of a context for object recognition. Availability of large data sets like ImageNet and VGG has provided scopes of applying machine learning classifiers to train models. However high data dimensionality is an issue while training classifiers such as Support Vector Machine (SVM) and perceptron. To reduce data dimensionality and take advantage of parallel and distributed processing, we propose a framework with Convolutional Neural Network (CNN) as Feature extractor and SVM and perceptron as the classifier. MPI (Message passing interface) was used for programming clusters of CPUs. SVM showed 1.05x times improvement over perceptron in terms of run time and CNN reduced data dimensionality by 10x times.
PR-144: SqueezeNext: Hardware-Aware Neural Network DesignJinwon Lee
Tensorfkow-KR 논문읽기모임 PR12 144번째 논문 review입니다.
이번에는 Efficient CNN의 대표 중 하나인 SqueezeNext를 review해보았습니다. SqueezeNext의 전신인 SqueezeNet도 같이 review하였고, CNN을 평가하는 metric에 대한 논문인 NetScore에서 SqueezeNext가 1등을 하여 NetScore도 같이 review하였습니다.
논문링크:
SqueezeNext - https://arxiv.org/abs/1803.10615
SqueezeNet - https://arxiv.org/abs/1602.07360
NetScore - https://arxiv.org/abs/1806.05512
영상링크: https://youtu.be/WReWeADJ3Pw
PR095: Modularity Matters: Learning Invariant Relational Reasoning TasksJinwon Lee
Tensorflow-KR 논문읽기모임 95번째 발표영상입니다
Modularity Matters라는 제목으로 visual relational reasoning 문제를 풀 수 있는 방법을 제시한 논문입니다. 기존 CNN들이 이런 문제이 취약함을 보여주고 이를 해결하기 위한 방법을 제시합니다. 관심있는 주제이기도 하고 Bengio 교수님 팀에서 쓴 논문이라서 review 해보았습니다
발표영상: https://youtu.be/dAGI3mlOmfw
논문링크: https://arxiv.org/abs/1806.06765
TensorFlow Korea 논문읽기모임 PR12 243째 논문 review입니다
이번 논문은 RegNet으로 알려진 Facebook AI Research의 Designing Network Design Spaces 입니다.
CNN을 디자인할 때, bottleneck layer는 정말 좋을까요? layer 수는 많을 수록 높은 성능을 낼까요? activation map의 width, height를 절반으로 줄일 때(stride 2 혹은 pooling), channel을 2배로 늘려주는데 이게 최선일까요? 혹시 bottleneck layer가 없는 게 더 좋지는 않은지, 최고 성능을 내는 layer 수에 magic number가 있는 건 아닐지, activation이 절반으로 줄어들 때 channel을 2배가 아니라 3배로 늘리는 게 더 좋은건 아닌지?
이 논문에서는 하나의 neural network을 잘 design하는 것이 아니라 Auto ML과 같은 기술로 좋은 neural network을 찾을 수 있는 즉 좋은 neural network들이 살고 있는 좋은 design space를 design하는 방법에 대해서 얘기하고 있습니다. constraint이 거의 없는 design space에서 human-in-the-loop을 통해 좋은 design space로 그 공간을 좁혀나가는 방법을 제안하였는데요, EfficientNet보다 더 좋은 성능을 보여주는 RegNet은 어떤 design space에서 탄생하였는지 그리고 그 과정에서 우리가 당연하게 여기고 있었던 design choice들이 잘못된 부분은 없었는지 아래 동영상에서 확인하실 수 있습니다~
영상링크: https://youtu.be/bnbKQRae_u4
논문링크: https://arxiv.org/abs/2003.13678
PR-231: A Simple Framework for Contrastive Learning of Visual RepresentationsJinwon Lee
TensorFlow Korea 논문읽기모임 PR12 231번째 논문 review 입니다
이번 논문은 Google Brain에서 나온 A Simple Framework for Contrastive Learning of Visual Representations입니다. Geoffrey Hinton님이 마지막 저자이시기도 해서 최근에 더 주목을 받고 있는 논문입니다.
이 논문은 최근에 굉장히 핫한 topic인 contrastive learning을 이용한 self-supervised learning쪽 논문으로 supervised learning으로 학습한 ResNet50와 동일한 성능을 얻을 수 있는 unsupervised pre-trainig 방법을 제안하였습니다. Data augmentation, Non-linear projection head, large batch size, longer training, NTXent loss 등을 활용하여 훌륭한 representation learning이 가능함을 보여주었고, semi-supervised learning이나 transfer learning에서도 매우 뛰어난 결과를 보여주었습니다. 자세한 내용은 영상을 참고해주세요
논문링크: https://arxiv.org/abs/2002.05709
영상링크: https://youtu.be/FWhM3juUM6s
PR-284: End-to-End Object Detection with Transformers(DETR)Jinwon Lee
TensorFlow Korea 논문읽기모임 PR12 284번째 논문 review입니다.
이번 논문은 Facebook에서 나온 DETR(DEtection with TRansformer) 입니다.
arxiv-sanity에 top recent/last year에서 가장 상위에 자리하고 있는 논문이기도 합니다(http://www.arxiv-sanity.com/top?timefilter=year&vfilter=all)
최근에 ICLR 2021에 submit된 ViT로 인해서 이제 Transformer가 CNN을 대체하는 것 아닌가 하는 얘기들이 많이 나오고 있는데요, 올 해 ECCV에 발표된 논문이고 feature extraction 부분은 CNN을 사용하긴 했지만 transformer를 활용하여 효과적으로 Object Detection을 수행하는 방법을 제안한 중요한 논문이라고 생각합니다. 이 논문에서는 detection 문제에서 anchor box나 NMS(Non Maximum Supression)와 같은 heuristic 하고 미분 불가능한 방법들이 많이 사용되고, 이로 인해서 유독 object detection 문제는 딥러닝의 철학인 end-to-end 방식으로 해결되지 못하고 있음을 지적하고 있습니다. 그 해결책으로 bounding box를 예측하는 문제를 set prediction problem(중복을 허용하지 않고, 순서에 무관함)으로 보고 transformer를 활용한 end-to-end 방식의 알고리즘을 제안하였습니다. anchor box도 필요없고 NMS도 필요없는 DETR 알고리즘의 자세한 내용이 알고싶으시면 영상을 참고해주세요!
영상링크: https://youtu.be/lXpBcW_I54U
논문링크: https://arxiv.org/abs/2005.12872
PR-270: PP-YOLO: An Effective and Efficient Implementation of Object DetectorJinwon Lee
TensorFlow Korea 논문읽기모임 PR12 270번째 논문 review입니다.
이번 논문은 Baidu에서 나온 PP-YOLO: An Effective and Efficient Implementation of Object Detector입니다. YOLOv3에 다양한 방법을 적용하여 매우 높은 성능과 함께 매우 빠른 속도 두마리 토끼를 다 잡아버린(?) 그런 논문입니다. 논문에서 사용한 다양한 trick들에 대해서 좀 더 깊이있게 살펴보았습니다. Object detection에 사용된 기법 들 중에 Deformable convolution, Exponential Moving Average, DropBlock, IoU aware prediction, Grid sensitivity elimination, MatrixNMS, CoordConv, 등의 방법에 관심이 있으시거나 알고 싶으신 분들은 영상과 발표자료를 참고하시면 좋을 것 같습니다!
논문링크: https://arxiv.org/abs/2007.12099
영상링크: https://youtu.be/7v34cCE5H4k
In machine learning, a convolutional neural network is a class of deep, feed-forward artificial neural networks that have successfully been applied fpr analyzing visual imagery.
[2020 CVPR Efficient DET 논문리뷰]
안녕하세요 딥러닝 논문읽기 모임입니다.
오늘 소개드릴 논문은 2020 CVPR에서 발표된, Efficient Net 저자가 발표한 'Efficient DET'입니다. 제목에서 유추 가능하듯 Backbone을 Efficient Net으로 사용하여 Object Detection Task에 적용 했다는 점을 유추할 수 있습니다.
해당 논문은 위 사실을 제외하고, 조금 특별한 Feature Pyramid Network를 적용하여 더욱 성능적으로 향상을 시켰는대요.
이러한 내용을 바탕으로 아직까지도 paperswithcode에 상위권에 랭크되어 있는 논문 입니다.
오늘 논문 이미지처리팀 이찬혁님이 자세하고 디테일한 리뷰를 도와주셨습니다.
많은 관심 미리 감사드립니다!
https://youtu.be/Mq4aqDgZ2bI
Explores the type of structure learned by Convolutional Neural Networks, the applications where they're most valuable and a number of appropriate mental models for understanding deep learning.
In this presentation we discuss the convolution operation, the architecture of a convolution neural network, different layers such as pooling etc. This presentation draws heavily from A Karpathy's Stanford Course CS 231n
A Framework for Scene Recognition Using Convolutional Neural Network as Featu...Tahmid Abtahi
Scene recognition is one of the hallmark tasks of computer vision, allowing definition of a context for object recognition. Availability of large data sets like ImageNet and VGG has provided scopes of applying machine learning classifiers to train models. However high data dimensionality is an issue while training classifiers such as Support Vector Machine (SVM) and perceptron. To reduce data dimensionality and take advantage of parallel and distributed processing, we propose a framework with Convolutional Neural Network (CNN) as Feature extractor and SVM and perceptron as the classifier. MPI (Message passing interface) was used for programming clusters of CPUs. SVM showed 1.05x times improvement over perceptron in terms of run time and CNN reduced data dimensionality by 10x times.
BalloonNet: A Deploying Method for a Three-Dimensional Wireless Network Surro...Naoki Shibata
Aiming at fast establishment of a wireless network around a multi-level building in a disaster area, we propose an efficient method to determine the locations of network nodes in the air. Nodes are attached to balloons outside a building and deployed in the air so that the network can be accessed from anywhere in the building. In this paper, we introduce an original radio propagation model for predicting path loss from an outdoor position to a position inside a building. In order to address the three-dimensional deployment problem, the proposed method optimizes an objective function for satisfying two goals: (1) guarantee the coverage: the target space needs to be covered by over a certain percentage by wireless network nodes, (2) minimize the number of network nodes. For solving this problem, we propose an algorithm based on a genetic algorithm. To evaluate the proposed method, we compared our method with three benchmark methods, and the results show that the proposed method requires fewer nodes than other methods.
A neural network is a network or circuit of neurons.
The neural network has layers of units where each layer takes some value from the previous layer.
That way, systems that are based on neural network can
compute inputs to get the needed output.
The same way neurons pass signals around the brain, and values
are passed from one unit in an artificial neural network to another
to perform the required computation and get new value as output.
The united are layers, forming a system that starts from the layers used for imputing to layer that is used to provide the output
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Building RAG with self-deployed Milvus vector database and Snowpark Container...Zilliz
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
6. DEEP LEARNING TO NAS
Neural Architecture Search(NAS)?
Design
an Individual Network
→
Design
a Network Generator
Automation of
Feature Engineering
→
Automation of
Architecture Engineering
10. • Network Generator 𝑔
𝑔 ∶ Θ ↦ 𝒩
where Θ : a parameter space, 𝒩 : a family of related networks
• e.g. In ResNet generator, 𝒩: ResNets and 𝜃 ∈ Θ specify the number of
stages, number of residual blocks for each stage, depth/width/filter sizes,
activation types, etc.
• Deterministic
• Stochastic network generator 𝑔
𝑔 ∶ Θ × 𝑆 ↦ 𝒩
where Θ : a parameter space, 𝑆 : seeds of a pseudo-random
number generator, 𝒩 : a family of related networks
• e.g. NAS. 𝜃: weight matrices of LSTM, The output of each LSTM time-
step is a probability distribution conditioned on 𝜃
STOCHASTIC NETWORK GENERATOR
Randomly Wired Neural Networks
11. • Turing’s unorganized machines, which is a form of the earliest
randomly connected neural network
• Infant human’s cortex: Small-world properties
• Random graph modeling has been used as a tool to study the neural
networks of human brains
• Random graph models are an effective tool for modeling and
analyzing real-world graphs, e.g., social networks, world wide web,
citation networks
MOTIVATION
Randomly Wired Neural Networks
12. 1. Generating general graphs(DAG)
2. Mapping from a general graph to neural network
operations
• Edge operations
- Data flow
• Node operations
- Aggregation: the input data via a weighted sum (learnable
and positive)
- Transformation: ReLU-convolution-BN triplet
- Distribution: The same copy of the transformed data
is sent out
3. Attaching Input and Output nodes
4. Stages
METHODOLOGY
Randomly Wired Neural Networks
13. • Additive aggregation maintains the same number of output channels
as input channels.
• Transformed data can be combined with the data from any other
nodes.
• Fixing the channel count keeps the FLOPs and parameter count
unchanged for each node, regardless of its input and output degrees.
• The overall FLOPs and parameter count of a graph are roughly
proportional to the number of nodes and nearly independent of the
number of edges
• This enables the comparison of different graphs without
inflating/deflating model complexity. Differences in task
performance are therefore reflective of the properties of the
wiring pattern.
NICE PROPERTIES OF NODE OPERATION
Randomly Wired Neural Networks
14. • Input
• The same copy of the data flow
• Output
• average(unweighted) from all original
output nodes.
ATTACHING INPUT AND OUTPUT NODES
Randomly Wired Neural Networks
Extra input node
Extra output node
15. • An entire network consists of multiple stages.
• One random graph represents one stage
• For all nodes that are directly connected to the input node, their
transformations are modified to have a stride 2.
• The channel count in a random graph is increased by 2x when going
from one stage to the next stage.
STAGES
Randomly Wired Neural Networks
RandWire Architecture
17. • Erdös-Rényi(ER), 1959.
• ER(N, P)
• Has N nodes
• An edge between two nodes is connected with probability P.
• The ER generation model has only a single parameter P, and is denoted
as ER(P).
• Any graph with N nodes has non-zero probability of being generated by
the ER model.
GENERATING GENERAL GRAPHS
Randomly Wired Neural Networks
18. • Barabási-Albert (BA), 1999.
• BA(N, M)
• 1 ≤ M < N
𝑖𝑛𝑖𝑡𝑖𝑎𝑡𝑒 𝑡ℎ𝑒 𝑔𝑟𝑎𝑝ℎ 𝐺 𝑎𝑠 𝑀 𝑛𝑜𝑑𝑒𝑠 𝑤𝑖𝑡ℎ𝑜𝑢𝑡 𝑎𝑛𝑦 𝑒𝑑𝑔𝑒
𝑖𝑡𝑒𝑟𝑎𝑡𝑒: 𝐴𝑑𝑑 𝑎 𝑛𝑜𝑑𝑒 𝑣 𝑡 𝑠. 𝑡.
𝑓𝑜𝑟 𝑛𝑜𝑑𝑒 𝑣 𝑖𝑛 𝐺
𝑐𝑜𝑛𝑛𝑒𝑐𝑡 𝑣 𝑎𝑛𝑑 𝑣 𝑡 𝑏𝑦
𝑃 𝑣 𝑡 𝑎𝑛𝑑 𝑣 𝑎𝑟𝑒 𝑐𝑜𝑛𝑛𝑒𝑐𝑡𝑒𝑑 ∝ 𝑑𝑒𝑔𝑟𝑒𝑒(𝑣)
𝑢𝑛𝑡𝑖𝑙 𝑣 𝑡 ℎ𝑎𝑠 𝑀 𝑒𝑑𝑔𝑒𝑠
𝑢𝑛𝑡𝑖𝑙 𝐺 ℎ𝑎𝑠 𝑁 𝑛𝑜𝑑𝑒𝑠
• has exactly M(N-M) edges. → a subset of all graphs with N nodes
GENERATING GENERAL GRAPHS
Randomly Wired Neural Networks
19. • Watts-Strogatz(WS), 1998.
• WS(N, K, P)
• “Small World” model.
High clustering, small diameter
0. N nodes를 원형으로 나열
1. 각 node 별로 양쪽으로 K/2개의 nodes를 연결
2. 시계방향으로 돌면서 rewire with probability P (uniformly)
• Has N⋅K edges → Smaller subset of N-node Graph
GENERATING GENERAL GRAPHS
Randomly Wired Neural Networks
20. • a stochastic network generator 𝑔(𝜃, 𝑠).
• The random graph parameters, P, M, (K; P) in ER, BA, WS
respectively, are part of the parameters 𝜃.
• The “optimization” of such a 1-or 2-parameter space is essentially
done by trial-and-error by human designers. – line/grid search
• The accuracy variation of our networks is small for different seeds 𝑠 so
they perform no random search and report mean accuracy of
multiple random network instances.
DESIGN AND OPTIMIZATION
Randomly Wired Neural Networks
22. • Imagenet Classification
• A small computation regime – MobileNet& ShuffleNet
• A regular computation regime – ResNet-50/101
• N nodes, C channels determine network complexity.
• N = 32, C = 79 for the small regime.
• N = 32, C = 109 or 154 for the regular regime.
• Random Seeds
• Randomlysample 5 network instances, train them from scratch.
• Report the classification accuracy with “mean±std” for all 5 network instances.
• Implementation Details
• Train for 100 epochs
• Half-period-cosine shaped learning rate decay and initial learning rate 0.1
• The weight decay is 5e-5
• Momentum 0.9
• Label smoothing regularization with a coefficient of 0.1
ARCHITECTURE DETAILS
Experiments
23. • 모두 학습 성공
• ER, BA, WS 모두 특정 세팅에서 mean accuracy > 73%
• Accuracy의 variance가 작음(std : 0.2 ~ 0.4 %)
• Random generator 별로 평균적인 accuracy차이가 있음
IMAGENET CLASSIFICATION
Experiments
24. • Node remove
• WS
the mean degradation of accuracy is larger when the output degree
of the removed node is higher.
“hub” nodes in WS that send information to many nodes are
influential
GRAPH DAMAGE
Experiments
25. • Edge remove
• If the input degree of an edge’s target node is smaller, removing this
edge tends to change a larger portion of the target node’s inputs.
• ER
less sensitive to edge removal, possibly because in ER’s definition
wiring of every edge is independent.
GRAPH DAMAGE
Experiments
26. • Same conv in all nodes
• adjust the factor C to keep the complexity of all alternative networks
• the Pearson correlation between any two series in Figure is 0:91 ~
0:98
NODE OPERATIONS
Experiments
27. • Small computation regime
COMPARISONS
Experiments
*250 epochs for fair comparisons
28. • Regular computation regime
• Use a regularization method inspired by edge removal analysis.
Randomly remove one edge whose target node has input degree > 1
with probability of 0.1.
COMPARISONS
Experiments
29. • Larger computation
• Increase the test image size to 320 x 320 without retraining
COMPARISONS
Experiments
30. • Object detection
• The features learned by randomly wired networks can also transfer.
COMPARISONS
Experiments
32. • The mean accuracy of these models is competitive with hand-
designed and optimized from NAS(Net).
• The authors hope that future work exploring new generator designs
may yield new, powerful networks designs.
• Contribution
• Layer type보다는 wiring pattern에 집중하여 search space를 잘 정의함
• 좋은 Search space를 찾는 것 만으로도 좋은 결과를 낼 수 있음
• (Stochastic) Network Generator 개념을 도입함
CONCLUSION
34. • Search Space
• 더 나은 search space를 찾는 아이디어
• Search Methods
• Prior knowledge: 의도나 해석이 없음
• 성능이 좋은 Network의 특성에 대한 연구가 있으면 좋을듯(AutoML의 문
제)
DISCUSSION
35. [1] Xie, S., Kirillov, A., Girshick, R., & He, K. (2019). Exploring Randomly
Wired Neural Networks for Image Recognition. arXiv preprint
arXiv:1904.01569.
[2] Elsken, T., Metzen, J. H., & Hutter, F. (2019). Neural Architecture
Search: A Survey. Journal of Machine Learning Research, 20(55), 1-21.
[3] Zoph, B., & Le, Q. V. (2017). Neural architecture search with
reinforcement learning. ICLR 2017.
[4] Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018). Learning
transferable architectures for scalable image recognition. In
Proceedings of the IEEE conference on computer vision and pattern
recognition (pp. 8697-8710).
[5] Jinwon, L. PR-155: Exploring Randomly Wired Neural Networks for
Image Recognition. https://www.youtube.com/watch?v=qnGm1h365tc
REFERENCES
Deep learning은 Feature engineering을 자동화하는데 기여함
하지만 이는 곧 직접 network architecture를 구성하는 Architecture engineering으로 변질됨
많은 network architecture들이 개발됐지만 time-consuming한 작업이고 error-prune한 작업임
Layer 길이가 7인 DNN등 space를 제한 가능. 사전 지식(conv쓰는게 좋음, 3x3 conv가 좋음, BN쓰면 좋음 등)을 반영하면 search space를 줄일 수 있어 효율적일 수 있지만 이 또한 novel architecture 탐색을 방해하는 human bias가 될 수 있음. 좋은 예시: Cell/Block을 학습 시켜서 반복시키면 다른 data에 transfer가능하면서도 space reduction하면서 좋은 성능 유지 가능.
RL, Random pick, evolutionary methods 등 사용 가능. Search space는 보통 exponentially 크거나 unbounded -> 잘 찾아야함; exploration-exploitation trade-off
보통은 그냥 training-validation을 거치는데 최근 이 과정을 효율적으로 만들기 위한 시도가 있음
“Connectionist” Approach
Swish와 같은 activation function이나
Auto augment와 같은 augmentation도 찾아내곤 함
결과적으로 한 형태의 convolution이나 layer크기들만 사용하게 됨
지금까지의 works는 search space에 대한 조절을 layer위주로 하거나 search strategy 위주의 연구가 많았음. 저자는 wiring에 대한 영향이 궁금함
(이제부터 NAS는 Neural Architecture Search with Reinforcement Learning)
Relu를 마지막에 두면 weight가 positive여서 positive 가 계속 더해지게 돼서 값이 계속 커짐-> BN을 마지막에 둬서 조절함
Issue: 특별한 형태의 확률이 낮고 평균적으로 비슷한 그래프를 만들듯
P > ln(N)/N 이면 single component(connected)
This gives one example on how an underlying prior can be introduced by the graph generator in spite of randomness.
친구가 많은 애들이랑 연결될 확률이 높음
“Rewiring” is defined as uniformly choosing a random node that is not v and that is not a duplicate edge.
We randomly remove one
node (top) or remove one edge (bottom) from a graph after the
network is trained, and evaluate the loss (delta) in accuracy on ImageNet.
Red circle: mean; gray bar: median; orange box: interquartile range;
blue dot: an individual damaged instance.
We randomly remove one
node (top) or remove one edge (bottom) from a graph after the
network is trained, and evaluate the loss (delta) in accuracy on ImageNet.
Red circle: mean; gray bar: median; orange box: interquartile range;
blue dot: an individual damaged instance.
Regularization의 효과를 비교하는 실험도 했으면 좋겠다
그런데 내가 재현하기가 너무 어렵다