The document discusses lower bounds for sorting algorithms. It shows that the worst case and average case lower bounds for sorting are both Ω(n log n). This lower bound is derived by analyzing the minimum depth of balanced binary decision trees required to represent the n! possible permutations of n elements. Several algorithms, including heapsort, are shown to achieve the optimal time complexity of O(n log n), matching this lower bound. Problem transformations are also introduced as a method for proving lower bounds.
We trained a large, deep convolutional neural network to classify the 1.2 million
high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif-
ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5%
and 17.0% which is considerably better than the previous state-of-the-art. The
neural network, which has 60 million parameters and 650,000 neurons, consists
of five convolutional layers, some of which are followed by max-pooling layers,
and three fully-connected layers with a final 1000-way softmax. To make train-
ing faster, we used non-saturating neurons and a very efficient GPU implemen-
tation of the convolution operation. To reduce overfitting in the fully-connected
layers we employed a recently-developed regularization method called “dropout”
that proved to be very effective. We also entered a variant of this model in the
ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%,
compared to 26.2% achieved by the second-best entry.
Word embedding, Vector space model, language modelling, Neural language model, Word2Vec, GloVe, Fasttext, ELMo, BERT, distilBER, roBERTa, sBERT, Transformer, Attention
We trained a large, deep convolutional neural network to classify the 1.2 million
high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif-
ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5%
and 17.0% which is considerably better than the previous state-of-the-art. The
neural network, which has 60 million parameters and 650,000 neurons, consists
of five convolutional layers, some of which are followed by max-pooling layers,
and three fully-connected layers with a final 1000-way softmax. To make train-
ing faster, we used non-saturating neurons and a very efficient GPU implemen-
tation of the convolution operation. To reduce overfitting in the fully-connected
layers we employed a recently-developed regularization method called “dropout”
that proved to be very effective. We also entered a variant of this model in the
ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%,
compared to 26.2% achieved by the second-best entry.
Word embedding, Vector space model, language modelling, Neural language model, Word2Vec, GloVe, Fasttext, ELMo, BERT, distilBER, roBERTa, sBERT, Transformer, Attention
Attention Is All You Need.
With these simple words, the Deep Learning industry was forever changed. Transformers were initially introduced in the field of Natural Language Processing to enhance language translation, but they demonstrated astonishing results even outside language processing. In particular, they recently spread in the Computer Vision community, advancing the state-of-the-art on many vision tasks. But what are Transformers? What is the mechanism of self-attention, and do we really need it? How did they revolutionize Computer Vision? Will they ever replace convolutional neural networks?
These and many other questions will be answered during the talk.
In this tech talk, we will discuss:
- A piece of history: Why did we need a new architecture?
- What is self-attention, and where does this concept come from?
- The Transformer architecture and its mechanisms
- Vision Transformers: An Image is worth 16x16 words
- Video Understanding using Transformers: the space + time approach
- The scale and data problem: Is Attention what we really need?
- The future of Computer Vision through Transformers
Speaker: Davide Coccomini, Nicola Messina
Website: https://www.aicamp.ai/event/eventdetails/W2021101110
word2vec, LDA, and introducing a new hybrid algorithm: lda2vec👋 Christopher Moody
Available with notes:
http://www.slideshare.net/ChristopherMoody3/word2vec-lda-and-introducing-a-new-hybrid-algorithm-lda2vec
(Data Day 2016)
Standard natural language processing (NLP) is a messy and difficult affair. It requires teaching a computer about English-specific word ambiguities as well as the hierarchical, sparse nature of words in sentences. At Stitch Fix, word vectors help computers learn from the raw text in customer notes. Our systems need to identify a medical professional when she writes that she 'used to wear scrubs to work', and distill 'taking a trip' into a Fix for vacation clothing. Applied appropriately, word vectors are dramatically more meaningful and more flexible than current techniques and let computers peer into text in a fundamentally new way. I'll try to convince you that word vectors give us a simple and flexible platform for understanding text while speaking about word2vec, LDA, and introduce our hybrid algorithm lda2vec.
Attention Is All You Need.
With these simple words, the Deep Learning industry was forever changed. Transformers were initially introduced in the field of Natural Language Processing to enhance language translation, but they demonstrated astonishing results even outside language processing. In particular, they recently spread in the Computer Vision community, advancing the state-of-the-art on many vision tasks. But what are Transformers? What is the mechanism of self-attention, and do we really need it? How did they revolutionize Computer Vision? Will they ever replace convolutional neural networks?
These and many other questions will be answered during the talk.
In this tech talk, we will discuss:
- A piece of history: Why did we need a new architecture?
- What is self-attention, and where does this concept come from?
- The Transformer architecture and its mechanisms
- Vision Transformers: An Image is worth 16x16 words
- Video Understanding using Transformers: the space + time approach
- The scale and data problem: Is Attention what we really need?
- The future of Computer Vision through Transformers
Speaker: Davide Coccomini, Nicola Messina
Website: https://www.aicamp.ai/event/eventdetails/W2021101110
word2vec, LDA, and introducing a new hybrid algorithm: lda2vec👋 Christopher Moody
Available with notes:
http://www.slideshare.net/ChristopherMoody3/word2vec-lda-and-introducing-a-new-hybrid-algorithm-lda2vec
(Data Day 2016)
Standard natural language processing (NLP) is a messy and difficult affair. It requires teaching a computer about English-specific word ambiguities as well as the hierarchical, sparse nature of words in sentences. At Stitch Fix, word vectors help computers learn from the raw text in customer notes. Our systems need to identify a medical professional when she writes that she 'used to wear scrubs to work', and distill 'taking a trip' into a Fix for vacation clothing. Applied appropriately, word vectors are dramatically more meaningful and more flexible than current techniques and let computers peer into text in a fundamentally new way. I'll try to convince you that word vectors give us a simple and flexible platform for understanding text while speaking about word2vec, LDA, and introduce our hybrid algorithm lda2vec.
Recommendations for the Shear Assessment of Reinforced Concrete Solid Slab Br...Eva Lantsoght
As a result of the heavier live load models and more conservative shear approaches prescribed by the recently implemented Eurocodes, a large number of existing reinforced concrete solid slab bridges in the Netherlands were found to be shear-critical. The beneficial effect of the transverse load redistribution in slabs under concentrated loads is not taken into account. To quantify this effect, a comprehensive number of experiments was carried out. These results are used to formulate recommendations for the assessment practice for the case of solid slab bridges. The recommendations focus on the effective width over which the axle load can be distributed and its lower bound, the beneficial effect of transverse load redistribution and the influence of the yield strength of the reinforcement on the lower bound of the shear capacity. These recommendations are implemented in the “Quick Scan” method, leading to a significant reduction of the shear stresses.
The Cramér-Rao Lower Bound on the Variance of the Estimator of continuous SISO, MIMO and Digital Systems.
Probability and Estimation are prerequisite.
For comments please connect me at solo.hermelin@gmail.com.
For more presentations on different subjects visit my website at http://solohermelin.com.
Chap-2 Preliminary Concepts and Linear Finite Elements.pptxSamirsinh Parmar
non linear problem, linear finite element, formulate nonlinear problems, solve non linear problems, non linear structural problems, Linera vs Non Linear, Material Non linearity, Accuracy vs Convergence, Convergence Criteria
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
How to Create Map Views in the Odoo 17 ERPCeline George
The map views are useful for providing a geographical representation of data. They allow users to visualize and analyze the data in a more intuitive manner.
This is a presentation by Dada Robert in a Your Skill Boost masterclass organised by the Excellence Foundation for South Sudan (EFSS) on Saturday, the 25th and Sunday, the 26th of May 2024.
He discussed the concept of quality improvement, emphasizing its applicability to various aspects of life, including personal, project, and program improvements. He defined quality as doing the right thing at the right time in the right way to achieve the best possible results and discussed the concept of the "gap" between what we know and what we do, and how this gap represents the areas we need to improve. He explained the scientific approach to quality improvement, which involves systematic performance analysis, testing and learning, and implementing change ideas. He also highlighted the importance of client focus and a team approach to quality improvement.
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
Palestine last event orientationfvgnh .pptxRaedMohamed3
An EFL lesson about the current events in Palestine. It is intended to be for intermediate students who wish to increase their listening skills through a short lesson in power point.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
2. 2 -2
Lower bound
• Def : A lower bound of a problem is the least time
complexity required for any algorithm which can be used
to solve this problem.
☆ worst case lower bound
☆ average case lower bound
• The lower bound for a problem is not unique.
– e.g. Ω(1), Ω(n), Ω(n log n) are all lower bounds for
sorting.
– (Ω(1), Ω(n) are trivial)
3. 2 -3
• At present, if the highest lower bound of a problem
is Ω(n log n) and the time complexity of the best
algorithm is O(n2
).
– We may try to find a higher lower bound.
– We may try to find a better algorithm.
– Both of the lower bound and the algorithm may be
improved.
• If the present lower bound is Ω(n log n) and there
is an algorithm with time complexity O(n log n),
then the algorithm is optimal..
4. 2 -4
6 permutations for 3 data elements
a1
a2
a3
1 2 3
1 3 2
2 1 3
2 3 1
3 1 2
3 2 1
The worst case lower bound of sorting
8. 2 -8
Lower bound of sorting
• To find the lower bound, we have to find the
smallest depth of a binary tree.
• n! distinct permutations
n! leaf nodes in the binary decision tree.
• balanced tree has the smallest depth:
log(n!) = Ω(n log n)
lower bound for sorting: Ω(n log n)
9. 2 -9
Method 1:
log(n!) = log(n(n− 1)…1)
= log2 + log3 +…+ log n
> log xdx
n
1∫
= log e ln xdx
n
1∫
= log e[ ln ]x x x n
− 1
= log e(n ln n − n + 1)
= n log n − n log e + 1.44
≥ n log n − 1.44n
=Ω (n log n)
10. 2 -10
Method 2:
• Stirling approximation:
– n! ≈
– log n! ≈ log ≈ n log n ≈ Ω(n log n)
2πn
n
e
n
( )
2
1
2
π + +log logn n
n
e
n n! Sn
1 1 0.922
2 2 1.919
3 6 5.825
4 24 23.447
5 120 118.02
6 720 707.39
10 3,628,800 3,598,600
20 2.433x1018
2.423x1018
100 9.333x10157
9.328x10157
16. 2 -16
Implementation
• using a linear array
not a binary tree.
– The sons of A(h) are A(2h) and A(2h+1).
• time complexity: O(n log n)
17. 2 -17
Time complexity
Phase 1: construction
d = log n: depth
# of comparisons is at most:
L
d
=
−
∑
0
1
2(d−L)2L
=2d
L
d
=
−
∑
0
1
2L
−4
L
d
=
−
∑
0
1
L2L-1
(
L
k
=
∑
0
L2L-1
= 2k
(k−1)+1)
=2d(2d
−1) −4(2d-1
(d −1 −1) + 1)
:
= cn −2log n−4, 2 ≤c ≤4
d
L
d-L
18. 2 -18
Time complexity
Phase 2: output
2
i
n
=
−
∑
1
1
log i
= :
=2nlog n−4cn + 4, 2 ≤c ≤4
=O(n log n)
log i
i nodes
19. 2 -19
Average case lower bound of sorting
• By binary decision tree
• The average time complexity of a sorting algorithm:
the external path length of the binary tree
n!
• The external path length is minimized if the tree is
balanced.
(all leaf nodes on level d or level d−1)
20. 2 -20
unbalanced
external path length
= 4⋅3 + 1 = 13
balanced
external path length
= 2⋅3+3⋅2 = 12
Average case lower bound of sorting
21. 2 -21
Compute the min external path length
1. Depth of balanced binary tree with c leaf nodes:
d = log c
Leaf nodes can appear only on level d or d−1.
2. x1
leaf nodes on level d−1
x2
leaf nodes on level d
x1
+ x2
= c
x1
+ = 2d-1
⇒ x1
= 2d
−c
d-1
x2
2
23. 2 -23
Quicksort & Heapsort
• Quicksort is optimal in the average case.
(Θ(n log n) in average )
• (i)worst case time complexity of heapsort is
Θ(n log n)
(ii)average case lower bound: Ω(n log n)
– average case time complexity of heapsort is Θ(n log
n)
– Heapsort is optimal in the average case..
24. 2 -24
Improving a lower bound through
oracles
• Problem P: merge two sorted sequences A and B with
lengths m and n.
(1) Binary decision tree:
There are ways !
leaf nodes in the binary tree.
⇒ The lower bound for merging:
log ≤ m + n − 1 (conventional merging)
m n
n
+
m n
n
+
m n
n
+
25. 2 -25
When m = n
log
m n
n
+
= log
( )!
( !)
2
2
m
m
= log((2m)!) −2log m!
Using Stirling approximation
n! ≈2πn
n
e
n
( )
log
m n
n
+
≈2m −
1
2
log m + O(1)
Optimal algorithm: 2m −1 comparisons
log
m n
n
+
< 2m −1
26. 2 -26
(2) Oracle:
• The oracle tries its best to cause the algorithm
to work as hard as it might. (to give a very hard
data set)
• Sorted sequences:
– A: a1
< a2
< … < am
– B: b1
< b2
< … < bm
• The very hard case:
– a1
< b1
< a2
< b2
< … < am
< bm
27. 2 -27
• We must compare:
a1
: b1
b1
: a2
a2
: b2
:
bm-1
: am-1
am
: bm
• Otherwise, we may get a wrong result for some input data.
e.g. If b1
and a2
are not compared, we can not distinguish
a1
< b1
< a2
< b2
< … < am
< bm
and
a1
< a2
< b1
< b2
< … < am
< bm
• Thus, at least 2m−1 comparisons are required.
• The conventional merging algorithm is optimal for m = n.
28. 2 -28
Finding lower bound by problem
transformation
• Problem A reduces to problem B (A∝B)
– iff A can be solved by using any algorithm which
solves B.
– If A∝B, B is more difficult.
• Note: T(tr1
) + T(tr2
) < T(B)
T(A) ≤ T(tr1
) + T(tr2
) + T(B) ∼ O(T(B))
instance
of A
transformation
T(tr1)
instance of B
T(A) T(B) solver of B
answer
of A
transformation
T(tr2) answer of B
29. 2 -29
The lower bound of the convex
hull problem
• sorting ∝ convex hull
A B
• an instance of A: (x1
, x2
,…, xn
)
↓transformation
an instance of B: {( x1
, x1
2
), ( x2
, x2
2
),
…, ( xn
, xn
2
)}
assume: x1
< x2
< …< xn
30. 2 -30
• If the convex hull problem can be solved,
we can also solve the sorting problem.
– The lower bound of sorting: Ω(n log n)
• The lower bound of the convex hull
problem: Ω(n log n)
31. 2 -31
The lower bound of the Euclidean minimal
spanning tree (MST) problem
• sorting ∝ Euclidean MST
A B
• an instance of A: (x1
, x2
,…, xn
)
↓transformation
an instance of B: {( x1
, 0), ( x2
, 0),…, ( xn
, 0)}
– Assume x1
< x2
< x3
<…< xn
– ⇔there is an edge between (xi
, 0) and (xi+1
, 0) in
the MST, where 1 ≤ i ≤ n−1
32. 2 -32
• If the Euclidean MST problem can be
solved, we can also solve the sorting
problem.
– The lower bound of sorting: Ω(n log n)
• The lower bound of the Euclidean MST
problem: Ω(n log n)