Proc SQL can be used as an alternative to traditional SAS approaches like Proc Summary and data step merges for summarizing, joining, and selecting data. It allows sorting, using functions, and selecting values into macro variables in a single step. Proc SQL also supports creating indexes to improve efficiency of joins and performing many-to-many joins not possible with other methods.
Spark SQL Catalyst Code Optimization using Function Outlining with Kavana Bha...Databricks
Spark SQL Catalyst optimizer, post query plan optimization, compiles the SQL query to Java code. Without code generation, such query expressions would have to be interpreted for each row of data, by walking down a tree of nodes. This introduces large amounts of branches and virtual function calls that slow down execution. With code generation, a query is collapsed into a single optimized function that eliminates multiple function calls and leverages CPU registers for intermediate data.
This code is then compiled in runtime to Java bytecode using Janino compiler. This presentation focuses on further catalyst code generation optimizations possible using function outlining. Automatic code generation tools generally tend to generate huge optimized functions. Large functions that are frequently executed might degrade runtime performance by preventing JVM optimizations such as function inlining. To avoid this, code generation tools should try to contain independent logic into separate functions.
This presentation will take the audience through the Spark Catalyst Code generation, how automatic split of large functions into smaller functions was achieved and the performance benefits associated with it
This is a paper I wrote at Hotsos where we used Method-R and Trace Data to optimize performance. SQL tuning can be simple if you ask the right questions.
Learning
Base SAS,
Advanced SAS,
Proc SQl,
ODS,
SAS in financial industry,
Clinical trials,
SAS Macros,
SAS BI,
SAS on Unix,
SAS on Mainframe,
SAS interview Questions and Answers,
SAS Tips and Techniques,
SAS Resources,
SAS Certification questions...
visit http://sastechies.blogspot.com
No more struggles with Apache Spark workloads in productionChetan Khatri
Paris Scala Group Event May 2019, No more struggles with Apache Spark workloads in production.
Apache Spark
Primary data structures (RDD, DataSet, Dataframe)
Pragmatic explanation - executors, cores, containers, stage, job, a task in Spark.
Parallel read from JDBC: Challenges and best practices.
Bulk Load API vs JDBC write
An optimization strategy for Joins: SortMergeJoin vs BroadcastHashJoin
Avoid unnecessary shuffle
Alternative to spark default sort
Why dropDuplicates() doesn’t result consistency, What is alternative
Optimize Spark stage generation plan
Predicate pushdown with partitioning and bucketing
Why not to use Scala Concurrent ‘Future’ explicitly!
In this first of a series of presentations, we'll overview the differences between SQL and PL/SQL, and the first steps in optimization, as understanding RULE vs. COST, and how to slash 90% response time in data extractions running in SQL*Plus.
GridSQL is commonly thought of as a replication solution along the likes of Slony and Bucardo, but the open source GridSQL project actually allows PostgreSQL queries to be parallelized across many servers allowing performance to scale nearly linearly. In this session, we will discuss the advantages to using GridSQL for large multi-terabyte data warehouses and how to design your PostgreSQL schemas and queries to leverage GridSQL. We will dig into how GridSQL plans a query capable of spanning multiple PostgreSQL servers and executes across those nodes. We will delve into some performance expectations and where GridSQL should be deployed.
Learning
Base SAS,
Advanced SAS,
Proc SQl,
ODS,
SAS in financial industry,
Clinical trials,
SAS Macros,
SAS BI,
SAS on Unix,
SAS on Mainframe,
SAS interview Questions and Answers,
SAS Tips and Techniques,
SAS Resources,
SAS Certification questions...
visit http://sastechies.blogspot.com
Part 3 of the SQL Tuning workshop examines the different aspects of an execution plan, from cardinality estimates to parallel execution and explains what information you should be gleaming from the plan and how it affects the execution. It offers insight into what caused the Optimizer to make the decision it did as well as a set of corrective measures that can be used to improve each aspect of the plan.
Spark SQL Catalyst Code Optimization using Function Outlining with Kavana Bha...Databricks
Spark SQL Catalyst optimizer, post query plan optimization, compiles the SQL query to Java code. Without code generation, such query expressions would have to be interpreted for each row of data, by walking down a tree of nodes. This introduces large amounts of branches and virtual function calls that slow down execution. With code generation, a query is collapsed into a single optimized function that eliminates multiple function calls and leverages CPU registers for intermediate data.
This code is then compiled in runtime to Java bytecode using Janino compiler. This presentation focuses on further catalyst code generation optimizations possible using function outlining. Automatic code generation tools generally tend to generate huge optimized functions. Large functions that are frequently executed might degrade runtime performance by preventing JVM optimizations such as function inlining. To avoid this, code generation tools should try to contain independent logic into separate functions.
This presentation will take the audience through the Spark Catalyst Code generation, how automatic split of large functions into smaller functions was achieved and the performance benefits associated with it
This is a paper I wrote at Hotsos where we used Method-R and Trace Data to optimize performance. SQL tuning can be simple if you ask the right questions.
Learning
Base SAS,
Advanced SAS,
Proc SQl,
ODS,
SAS in financial industry,
Clinical trials,
SAS Macros,
SAS BI,
SAS on Unix,
SAS on Mainframe,
SAS interview Questions and Answers,
SAS Tips and Techniques,
SAS Resources,
SAS Certification questions...
visit http://sastechies.blogspot.com
No more struggles with Apache Spark workloads in productionChetan Khatri
Paris Scala Group Event May 2019, No more struggles with Apache Spark workloads in production.
Apache Spark
Primary data structures (RDD, DataSet, Dataframe)
Pragmatic explanation - executors, cores, containers, stage, job, a task in Spark.
Parallel read from JDBC: Challenges and best practices.
Bulk Load API vs JDBC write
An optimization strategy for Joins: SortMergeJoin vs BroadcastHashJoin
Avoid unnecessary shuffle
Alternative to spark default sort
Why dropDuplicates() doesn’t result consistency, What is alternative
Optimize Spark stage generation plan
Predicate pushdown with partitioning and bucketing
Why not to use Scala Concurrent ‘Future’ explicitly!
In this first of a series of presentations, we'll overview the differences between SQL and PL/SQL, and the first steps in optimization, as understanding RULE vs. COST, and how to slash 90% response time in data extractions running in SQL*Plus.
GridSQL is commonly thought of as a replication solution along the likes of Slony and Bucardo, but the open source GridSQL project actually allows PostgreSQL queries to be parallelized across many servers allowing performance to scale nearly linearly. In this session, we will discuss the advantages to using GridSQL for large multi-terabyte data warehouses and how to design your PostgreSQL schemas and queries to leverage GridSQL. We will dig into how GridSQL plans a query capable of spanning multiple PostgreSQL servers and executes across those nodes. We will delve into some performance expectations and where GridSQL should be deployed.
Learning
Base SAS,
Advanced SAS,
Proc SQl,
ODS,
SAS in financial industry,
Clinical trials,
SAS Macros,
SAS BI,
SAS on Unix,
SAS on Mainframe,
SAS interview Questions and Answers,
SAS Tips and Techniques,
SAS Resources,
SAS Certification questions...
visit http://sastechies.blogspot.com
Part 3 of the SQL Tuning workshop examines the different aspects of an execution plan, from cardinality estimates to parallel execution and explains what information you should be gleaming from the plan and how it affects the execution. It offers insight into what caused the Optimizer to make the decision it did as well as a set of corrective measures that can be used to improve each aspect of the plan.
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
Embracing GenAI - A Strategic ImperativePeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
Acetabularia Information For Class 9 .docxvaibhavrinwa19
Acetabularia acetabulum is a single-celled green alga that in its vegetative state is morphologically differentiated into a basal rhizoid and an axially elongated stalk, which bears whorls of branching hairs. The single diploid nucleus resides in the rhizoid.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
A Strategic Approach: GenAI in EducationPeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Home assignment II on Spectroscopy 2024 Answers.pdf
Save Coding Time with Proc SQL.ppt
1. 1
Efficient SAS Coding
with Proc SQL
When Proc SQL is Easier than Traditional
SAS Approaches
Mike Atkinson, May 4, 2005
2. 2
Note on Efficiency
There are at least two kinds of efficiency: SAS
program efficiency and SAS programmer efficiency.
This talk addresses the latter.
I think Proc SQL can reduce coding effort, once you
become familiar with SQL.
3. 3
Uses of Proc SQL
Proc SQL is the usual method to select data from an external
DBMS, such as Oracle, but don’t overlook that…
Proc SQL is very useful with SAS datasets:
• Summarising (alternative to Proc Summary)
• Select totals into a macro variables
• Joining (alternative to Merge in Data Step)
• Sorting
• Creating indexes for efficiency
4. 4
Selecting from an External DBMS
• Proc SQL is the usual method to select data from
an external DBMS, such as Oracle
• Can use pass-through SQL queries to allow
DBMS to optimise the query
However, this particular talk focuses on how Proc
SQL can be used with SAS datasets.
5. 5
Summarising Data
• Proc SQL can be used as an alternative to Proc
Summary
Advantages of Proc SQL:
• Can use functions (like put), removing need for an
additional data step
• Can join two or more datasets in the same step
• Don’t need to sort input prior to Proc SQL
• Can sort results in the same step
6. 6
Traditional SAS Code
proc sort data=sasdata.pracds
out=my_pracds;
by praclha;
run;
proc summary data=my_pracds;
by praclha;
output out=prac_lha_counts
(drop=_type_ rename=(_freq_=prac_cnt));
run;
7. 7
Proc SQL Code
proc sql;
create table prac_lha_counts as
select praclha,
count(*) as prac_cnt
from sasdata.pracds
group by praclha
order by praclha;
quit;
9. 9
Traditional SAS Code
proc sort data=sasdata.fitmserv
out=my_fitmserv;
by servcd;
run;
proc summary data=my_fitmserv;
by servcd;
output out=servcd_fitm_cnts_0
(drop=_type_ rename=(_freq_=fitm_cnt));
run;
data servcd_fitm_cnts;
set servcd_fitm_cnts_0;
servcd_descrip = put(servcd, svcd2ds.);
run;
10. 10
Proc SQL Code
proc sql;
create table servcd_fitm_cnts as
select servcd,
put(servcd, svcd2ds.) as servcd_descrip,
count(*) as fitm_cnt
from sasdata.fitmserv
group by servcd, servcd_descrip
order by servcd;
quit;
15. 15
Aside: Formatting Macro
Variables for Footnotes
%let pop_fmtd = %sysfunc(putn(&pop_lha61, comma9.));
footnote1 j=l “Note: LHA 61 had population of
&pop_fmtd in 2004/2005”;
Resulting footnote:
Note: LHA 61 had population of 208,372 in 2004/2005
16. 16
Joining SAS datasets
• Proc SQL can be an alternative to using a data
step merge
Advantages of Proc SQL:
• Datasets do not need to be sorted first
• Can use functions (like substr) in the join,
removing need for an additional data step
• Can perform many to many joins
• Can sort results in the same step
17. 17
Traditional SAS Code
proc sort data=my_pracs;
by pracnum;
run;
proc sort data=sasdata.pracds out=my_pracds;
by pracnum;
run;
data prac_info;
merge my_pracs (in=a keep=pracnum)
my_pracds;
by pracnum;
if (a);
run;
18. 18
Proc SQL Code
proc sql;
create table prac_info as
select a.pracnum, b.*
from my_pracs a
left join sasdata.pracds b
on a.pracnum = b.pracnum
order by a.pracnum;
quit;
19. 19
Alternative Proc SQL Code
(not quite the same)
proc sql;
create table prac_info as
select a.*
from sasdata.pracds a,
my_pracs b
where a.pracnum = b.pracnum
order by a.pracnum;
quit;
20. 20
Proc SQL Join
With Index for Efficiency
proc sql;
create unique index phnnum on my_phns;
quit;
proc sql;
create table hosp_ar_days as
select "2004/2005" as fiscal_year,
p.phnnum,
sum(h.ar_days) as ar_days
from sasdata.hosp04 h,
my_phns p
where h.phn = p.phnnum
group by fiscal_year, p.phnnum
order by p.phnnum;
quit;
22. 22
NOTE: MERGE statement has more than one
data set with repeats of BY values.
proc sort data=sasdata.fitmserv
out=my_fitmserv;
by fitm;
run;
proc sort data=sasdata.fitmspec
out=my_fitmspec;
by fitm;
run;
data spec_serv;
merge my_fitmserv (in=a)
my_fitmspec (in=b);
by fitm;
run;
23. 23
Many to Many Joins
Are Possible with Proc SQL
proc sql;
create table spec_serv as
select a.servcd,
b.speccode,
count(*) as fitm_count
from sasdata.fitmserv a,
sasdata.fitmspec b
where a.fitm = b.fitm
group by a.servcd, b.speccode;
quit;
25. 25
To Find Out, Are Any Fee Items Related to
More than One Service Code?
proc sql;
create table fitm_mults as
select fitm,
count(*) as rec_cnt
from sasdata.fitmserv
group by fitm
having count(*) > 1;
quit;
27. 27
Building a Format
proc sql noprint;
create table mypracnm as
select a.pracnum as start,
a.pracnum as end,
case when b.pracnum is missing then
put(a.pracnum, z5.) || ' - **No Name Found**'
else
put(a.pracnum, z5.) || ' - ' || b.pracnm
end as label,
'mypracnm' as fmtname,
'N' as type
from my_pracs a
left join sasdata.pracds b
on a.pracnum = b.pracnum;
quit;
proc format cntlin=mypracnm;
run;
28. 28
Sample Usage of Format
data two_small_pracs;
pracnum = 2600; output;
pracnum = 2624; output;
run;
proc print data=two_small_pracs;
format pracnum mypracnm.;
run;
Obs pracnum
1 02600 - **No Name Found**
2 02624 - SMITH JOHN M