Deskpool is a simple, professional and cost efficient VDI solution, which is very suitable for SMB market and really cheaper than PC. Deskpool is based on a open architecture and compatible with Xenserver, VMware ESXi, Hyper-V, vBox and KVM platforms.
Maximize Your Production Effort (Chinese)slantsixgames
Efficient Content Authoring Tools and Pipeline for Inter-Studio Asset Development
With the complexity of today's video games and their associated tight timelines, it is paramount for video game studios to have a highly efficient content authoring process and production workflow. With a trend towards outsourced development of game assets, there are additional considerations that are important for achieving optimal workflow between studios that are co-developing or sharing assets. This lecture gives valuable insight into how to create new content authoring tools and data transformation pipelines that promote efficient work flow for both internal and remote production teams. Specific considerations for outsourcing and worldwide development are made along the way.
Deskpool is a simple, professional and cost efficient VDI solution, which is very suitable for SMB market and really cheaper than PC. Deskpool is based on a open architecture and compatible with Xenserver, VMware ESXi, Hyper-V, vBox and KVM platforms.
Maximize Your Production Effort (Chinese)slantsixgames
Efficient Content Authoring Tools and Pipeline for Inter-Studio Asset Development
With the complexity of today's video games and their associated tight timelines, it is paramount for video game studios to have a highly efficient content authoring process and production workflow. With a trend towards outsourced development of game assets, there are additional considerations that are important for achieving optimal workflow between studios that are co-developing or sharing assets. This lecture gives valuable insight into how to create new content authoring tools and data transformation pipelines that promote efficient work flow for both internal and remote production teams. Specific considerations for outsourcing and worldwide development are made along the way.
1. Transfer learning involves using representations learned from one model or task to help train another model for a related but different task. This can speed up training and help address domain adaptation issues when the data distributions are different.
2. Self-supervised learning learns representations from unlabeled data using pretext tasks like predicting rotations or solving jigsaw puzzles. This pretraining helps models learn general visual features without human annotations.
3. Recent advances in contrastive self-supervised learning use data augmentations and large batches to learn representations by comparing an image to itself and making it more similar to itself than other images. This has achieved state-of-the-art results.
NTU DBME5028 Week5 Introduction to Machine Learning Sean Yu
This document provides an introduction and overview of machine learning. It discusses the core idea of machine learning, the general workflow of the machine learning process including defining the problem, collecting and cleaning data, selecting and building a model, evaluating key metrics, and creating presentations. It also covers common machine learning tasks like classification, data normalization, addressing overfitting, hyperparameter tuning, and evaluation metrics. The document uses examples from medical imaging to illustrate machine learning concepts and processes.
Practical aspects of medical image ai for hospital (IRB course)Sean Yu
Introduction of medical imaging AI, especially in digital pathology. The talk focused on how we come up with different projects, how to define the scope and challenges of these projects.
[Taiwan AI Academy] Machine learning and deep learning application examplesSean Yu
In the 40 minutes talk, we share what we've done in the data insight lab and the team project theta.
We've also shared about traps and tricks when encounter real world industrial cases.
1. Data generator in Python
for Keras (& Tensorflow)
Sean Yu, 2017/08/17
1
2. What is data generator and why we need it?
• 假設大家都寫過 keras or tensorflow
• Generally, we will have …
• training set / validation set / testing set
• We always load them ALL into memory
• MNIST – 60k images, 28 x 28 x 1
• Cifar10 – 60k images, 32 x 32 x 3
• EASY!
• However, if you have 100k+ 400 x 300 x 3 images,
it is impossible to load them all.
• We need to real-time load data
2
3. In GPU memory
In RAM (by CPU)
In Disk
Real-time data loading
• With real-time data loading, we can do many
manipulations on the data.
• Augmentation
• Add random noise
• …
• 你想對 data 幹什麼就幹什麼
• Principle of data generator
• A infinite loop that can ‘yield’ data when being requested
3
Prepared data
This batch
(for model)
Data in folder
…
4. Coding flow
4
Import basic libraries and
customized functions
Make data list & preload
Define data/image generator
Define model &
model related parameters
Train the model !! Evaluate the model
5. Today’s example data
• Kaggle, cats and dogs classification
• https://www.kaggle.com/c/dogs-vs-cats
• Keras blog 上面其實有類似的 example code 了, 改
寫一下而已
• Classification problem: cat or dog
• Training set: 25k
• Testing set 12.5k
5
7. Note
• Validation augmentation?
• We only need to augment it at begin (not dynamically!)
• 乾五郝? – 我的經驗, 好像有用捏
• 增加 validation 的難度 (複雜性) – 避免 validation 進步太
神速而太早被 earlystop
7
13. 其他 GPU 相關使用注意事項
• 在單張卡上, 限制每個 GPU memory fraction
• 在多張卡的機器上 (如 server), 選定使用特定
編號之 GPU
• 關閉 jupyter notebook 占用之 GPU 空間
13
- For Unix (python script)
In the terminal,
CUDA_VISIBLE_DEVICES=0 python
your_script.py
- For python script
At the script begin
Import os
os.environ[‘CUDA_VISIBLE_DEVICES’] = 0
- For jupyter notebook
At the notebook begin
%env CUDA_VISIBLE_DEVICES=0