Slides of the I Workshop on command-line tools with the collaboration of CAG (Center for Applied Genomics - Children's Hospital of Philadelphia) bioinformatics analysts.
1st day
Workshop on command line tools - day 2Leandro Lima
Slides of the I Workshop on command-line tools with the collaboration of CAG (Center for Applied Genomics - Children's Hospital of Philadelphia) bioinformatics analysts.
2nd day
Workshop on command line tools - day 2Leandro Lima
Slides of the I Workshop on command-line tools with the collaboration of CAG (Center for Applied Genomics - Children's Hospital of Philadelphia) bioinformatics analysts.
2nd day
Linux Shell Scripts and Shell Commands✌️Nazmul Hyder
I just shortly describe some Linux shell script and shell commands.Hopefully, it will help you to file edit, make, delete directory operations, grep, pipeline and lots of stuff to your Linux/Mac terminal.
Abstract:
This talk will introduce you to the concept of Kubernetes Volume plugins. We will not only help you understand the basic concepts, but more importantly, using practical examples, we will show how you can develop your own volume plugins and contribute them back to the community of the OSS project as large as Kubernetes.
We will conclude the talk by discussing various challenges one can come across when contributing to a high velocity OSS project of Kubernetes' size which can help you avoid the pain and enjoy the path.
Sched Link: http://sched.co/6BYB
This describes a Functional Programming approach to computing AWS Glacier "tree hash" values, hiding the tail-call elimination in Perl5 with a keyword and also shows how to accomplish the same result in Perl6.
This was the talk actually given at YAPC::NA 2016 by Dr. Conway and myself.
Linux Shell Scripts and Shell Commands✌️Nazmul Hyder
I just shortly describe some Linux shell script and shell commands.Hopefully, it will help you to file edit, make, delete directory operations, grep, pipeline and lots of stuff to your Linux/Mac terminal.
Abstract:
This talk will introduce you to the concept of Kubernetes Volume plugins. We will not only help you understand the basic concepts, but more importantly, using practical examples, we will show how you can develop your own volume plugins and contribute them back to the community of the OSS project as large as Kubernetes.
We will conclude the talk by discussing various challenges one can come across when contributing to a high velocity OSS project of Kubernetes' size which can help you avoid the pain and enjoy the path.
Sched Link: http://sched.co/6BYB
This describes a Functional Programming approach to computing AWS Glacier "tree hash" values, hiding the tail-call elimination in Perl5 with a keyword and also shows how to accomplish the same result in Perl6.
This was the talk actually given at YAPC::NA 2016 by Dr. Conway and myself.
Powerful and efficient tool leveraging UNIX commands for streamlined automation, enabling seamless execution of complex tasks and maximizing productivity.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Modern design is crucial in today's digital environment, and this is especially true for SharePoint intranets. The design of these digital hubs is critical to user engagement and productivity enhancement. They are the cornerstone of internal collaboration and interaction within enterprises.
Your Digital Assistant.
Making complex approach simple. Straightforward process saves time. No more waiting to connect with people that matter to you. Safety first is not a cliché - Securely protect information in cloud storage to prevent any third party from accessing data.
Would you rather make your visitors feel burdened by making them wait? Or choose VizMan for a stress-free experience? VizMan is an automated visitor management system that works for any industries not limited to factories, societies, government institutes, and warehouses. A new age contactless way of logging information of visitors, employees, packages, and vehicles. VizMan is a digital logbook so it deters unnecessary use of paper or space since there is no requirement of bundles of registers that is left to collect dust in a corner of a room. Visitor’s essential details, helps in scheduling meetings for visitors and employees, and assists in supervising the attendance of the employees. With VizMan, visitors don’t need to wait for hours in long queues. VizMan handles visitors with the value they deserve because we know time is important to you.
Feasible Features
One Subscription, Four Modules – Admin, Employee, Receptionist, and Gatekeeper ensures confidentiality and prevents data from being manipulated
User Friendly – can be easily used on Android, iOS, and Web Interface
Multiple Accessibility – Log in through any device from any place at any time
One app for all industries – a Visitor Management System that works for any organisation.
Stress-free Sign-up
Visitor is registered and checked-in by the Receptionist
Host gets a notification, where they opt to Approve the meeting
Host notifies the Receptionist of the end of the meeting
Visitor is checked-out by the Receptionist
Host enters notes and remarks of the meeting
Customizable Components
Scheduling Meetings – Host can invite visitors for meetings and also approve, reject and reschedule meetings
Single/Bulk invites – Invitations can be sent individually to a visitor or collectively to many visitors
VIP Visitors – Additional security of data for VIP visitors to avoid misuse of information
Courier Management – Keeps a check on deliveries like commodities being delivered in and out of establishments
Alerts & Notifications – Get notified on SMS, email, and application
Parking Management – Manage availability of parking space
Individual log-in – Every user has their own log-in id
Visitor/Meeting Analytics – Evaluate notes and remarks of the meeting stored in the system
Visitor Management System is a secure and user friendly database manager that records, filters, tracks the visitors to your organization.
"Secure Your Premises with VizMan (VMS) – Get It Now"
Why React Native as a Strategic Advantage for Startup Innovation.pdfayushiqss
Do you know that React Native is being increasingly adopted by startups as well as big companies in the mobile app development industry? Big names like Facebook, Instagram, and Pinterest have already integrated this robust open-source framework.
In fact, according to a report by Statista, the number of React Native developers has been steadily increasing over the years, reaching an estimated 1.9 million by the end of 2024. This means that the demand for this framework in the job market has been growing making it a valuable skill.
But what makes React Native so popular for mobile application development? It offers excellent cross-platform capabilities among other benefits. This way, with React Native, developers can write code once and run it on both iOS and Android devices thus saving time and resources leading to shorter development cycles hence faster time-to-market for your app.
Let’s take the example of a startup, which wanted to release their app on both iOS and Android at once. Through the use of React Native they managed to create an app and bring it into the market within a very short period. This helped them gain an advantage over their competitors because they had access to a large user base who were able to generate revenue quickly for them.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?XfilesPro
Worried about document security while sharing them in Salesforce? Fret no more! Here are the top-notch security standards XfilesPro upholds to ensure strong security for your Salesforce documents while sharing with internal or external people.
To learn more, read the blog: https://www.xfilespro.com/how-does-xfilespro-make-document-sharing-secure-and-seamless-in-salesforce/
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Accelerate Enterprise Software Engineering with PlatformlessWSO2
Key takeaways:
Challenges of building platforms and the benefits of platformless.
Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience.
How Choreo enables the platformless experience.
How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo.
Demo of an end-to-end app built and deployed on Choreo.
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...Hivelance Technology
Cryptocurrency trading bots are computer programs designed to automate buying, selling, and managing cryptocurrency transactions. These bots utilize advanced algorithms and machine learning techniques to analyze market data, identify trading opportunities, and execute trades on behalf of their users. By automating the decision-making process, crypto trading bots can react to market changes faster than human traders
Hivelance, a leading provider of cryptocurrency trading bot development services, stands out as the premier choice for crypto traders and developers. Hivelance boasts a team of seasoned cryptocurrency experts and software engineers who deeply understand the crypto market and the latest trends in automated trading, Hivelance leverages the latest technologies and tools in the industry, including advanced AI and machine learning algorithms, to create highly efficient and adaptable crypto trading bots
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Strategies for Successful Data Migration Tools.pptxvarshanayak241
Data migration is a complex but essential task for organizations aiming to modernize their IT infrastructure and leverage new technologies. By understanding common challenges and implementing these strategies, businesses can achieve a successful migration with minimal disruption. Data Migration Tool like Ask On Data play a pivotal role in this journey, offering features that streamline the process, ensure data integrity, and maintain security. With the right approach and tools, organizations can turn the challenge of data migration into an opportunity for growth and innovation.
Strategies for Successful Data Migration Tools.pptx
Workshop on command line tools - day 1
1. I Workshop on command-
line tools
(day 1)
Center for Applied Genomics
Children's Hospital of Philadelphia
February 12-13, 2015
2. Arguments
Come after the name of the program
Example:
cat file.txt (1 argument)
cut -f2 file.txt (2 arguments)
The number of spaces between arguments doesn't matter
cut -f2 file.txt
3. man - command manual
man <command>
man cat
man echo
man awk
4. which - which command is being called
which <command>
which cat
which echo
which awk
5. some tips (i)
Use <Tab> to auto-complete your commands or
file/directory names
To search old commands, you can use ↑ and ↓
arrows in your keyboard
6. some tips (ii)
The command history will return a list of your
last commands
Use ! to run the last command starting with…
Example:
!grep
This will run the last command starting with grep
7. Special characters (i)
^ : beginning of line
$ : end of line or beginning of variable name
? : any character (with one occurrence)
* : any character (with 0 or more occurrences)
# : start comments
[ ] : define sets of characters
8. Special characters (ii)
" " : define strings
' ' : define strings
- : start a parameter
` ` : define commands
; : separate commands
| : "pipe" commands
9. Special characters (iii)
~ : home directory
/ : separate internal directories
: escape character
n : new line (Linux)
r : new line (Mac)
t : tab
10. First steps
pwd # where am I?
whoami # who am I?
id <your_username> # what can I do?
date # what time/day is it?
11. cat - concatenate and print text files
cat file1.txt file2.txt > output.txt
cat *.bed > all.bed
cat -n : shows line numbers
cat -e : shows non-printing characters
12. echo - write to the standard output
echo Hello, CAG!
echo -e : prints escape characters
echo -e "CtAtG"
echo -e "CnAnG"
echo -n : prints and doesn't go to a new line
echo -n "CAG"; echo "123"
echo "CAG"; echo "123"
15. ls - list files in directories (i)
ls : list files of current directory
ls workshop : list files in directory workshop
ls -l : in long format
ls -t : list files sorted by time modified
ls -1 : force output to be one entry per line
ls -S : list files sorted by time modified
16. ls - list files in directories (ii)
ls -r : reverse the sorting
ls -a : list hidden files (which begin with a dot)
ls -h : show file size human-readable
ls -G : colors output
We can combine options:
ls -lhrt
17. ssh - secure shell (access remote servers) (i)
ssh <user>@<server>
ssh -t : exits after a list of commands
ssh limal@respublica.research.chop.edu
ssh limal@respublica.research.chop.edu -t top
ssh limal@respublica.research.chop.edu -t ls -lh
ssh limal@respublica.research.chop.edu -t ls -lh >
my_home_on_respub.txt
18. ssh - secure shell (access remote servers) (ii)
ssh -p <port> : access a specific port on server
ssh -X : open session with graphic/display options
(if you need to open a graphic program in a remote
server; e.g. IGV).
19. alias - "shortcut" for commands
alias <alias> : see what is a specific alias
alias ll # ll is not a real command. =)
alias resp='ssh limal@respublica.research.chop.edu'
resp
20. df - report file system disk space usage
df -h : human-readable
21. du - estimate file space usage
du -h : human-readable
22. mkdir - make directory
mkdir bioinfo_files
mkdir workshop_text_files
mkdir workshop123
mkdir -p 2015/February/12
# Suggestion:
# Create names that make sense
23. cd - change working directory
cd bioinfo_files
cd .. # go to directory above
cd ~ # go to home directory
cd - # go to previous directory
24. rmdir - remove empty directories
rmdir workshop123
rmdir 2015 # it will return an error
26. cp - copy files and directories
cp old_file.txt workshop_files
cp error.txt error_copy.txt
# To copy directories with its contents,
# use -r (recursive)
cp -r workshop_files bioinfo_files/
# Now, try...
cp -r workshop_files/ bioinfo_files/
27. scp - secure copy files and
directories in different servers
# Similar to "cp" (in this case, we're uploading)
scp *.txt limal@respublica.research.chop.edu:~/
# To copy directories with its contents,
# use -r (recursive)
scp -r w* limal@respublica.research.chop.edu:~/
# Downloading
scp limal@respublica.research.chop.edu:~/*.txt .
28. rm - remove files and directories
rm old_file.txt error_copy.txt
# Use -r (recursive) to remove
# directories and its contents
rm -r bioinfo_files/workshop_files/
rm -r 2015
29. ln - make links (pointers) of files
(it's good to avoid multiple copies)
# hard links keep the same if the original
# files are removed
ln workshop_files/old_file.txt hard.txt
# symbolic links break if the original
# files are removed
ln -s workshop_files/old_file.txt symbolic.txt
30. testing links
echo "hard" >> hard.txt
echo "symbolic" >> symbolic.txt
head hard.txt symbolic.txt
head workshop_files/old_file.txt
rm workshop_files/old_file.txt
head hard.txt symbolic.txt
32. tar - archiving
Create an archive:
tar -cvf newfile.tar file1 file2 dir1 dir2
tar -cvf BLs.tar bla.txt ble.txt blo.txt
tar -cvzf BLs.tar.gz bla.txt ble.txt blo.txt
Parameters: c (create), v (verbose), z (gzip), f (file)
33. tar - archiving
Extract from an archive:
tar -xvzf GWAS.tar.gz
tar -xvjf XHMM_results.tar.bz2
Parameters: x (extract), v (verbose), f (file),
z (gzip), j (bzip2)
34. gzip - zip files
ls -lh adhd.ped
gzip adhd.ped
ls -lh adhd.ped.gz
# to unzip, run "gunzip adhd.ped.gz"
35. zcat - cat for zipped files
zcat adhd.ped.gz # Ctrl+C to stop
36. less - file visualization
less DATA.xcnv
Use arrows (←↑→↓) to navigate the file
Type / to search
38. head - first lines
# first 20 lines
head -n 20 DATA.xcnv
# all lines, excluding last 2
# (on Linux, not Mac)
head -n -2 DATA.xcnv
39. tail - last lines
# last 20 lines
tail -n 20 DATA.xcnv
# from line 2 to the end
tail -n +2 DATA.xcnv
40. cut - get specific columns of file
# fields 1 to 3 and 6
cut -f 1-3,6 DATA.xcnv
# other examples
cut -f1 adhd.ped
cut -f1 -d' ' adhd.ped # delimiter = space
# other delimiters: comma, tab, etc.
cut -d, -f1-2 …
cut -d't' -f5,7,9 …
41. Using "|" (pipe) to join commands
cut -f 1-3,6 DATA.xcnv | head -n 1
cut -f 1-3,6 DATA.xcnv | less
zcat adhd.ped.gz | less
# Compare (same result? same time?)
zcat adhd.ped.gz | cut -f1 -d' ' | head
zcat adhd.ped.gz | head | cut -f1 -d' '
42. column - columnate lists
# using white spaces to separate
# and fill columns
column -t DATA.xcnv
column -s # choose separator
43. sort - sort lines of text files
sort DATA.xcnv
sort -k : choose specific field
sort -n : numeric-sort
sort -r : reverse
# Exercise: show 10 top CNVs with
# more targets (column 8)
44. uniq - report or filter out repeated lines in a file
cut -f1 DATA.xcnv | sort | uniq
# reporting counts of each line
cut -f5 DATA.xcnv | sort | uniq -c
45. wc - word, line, character and byte count
wc -l : number of lines
wc -w : number of words
wc -m : number of characters
cut -f5 DATA.xcnv | sort | uniq | wc -l
head -n1 DATA.xcnv | cut -f1 | wc -m
46. More exercises
1. What are the top 10 samples with more CNVs?
2. What are the top 5 largest CNVs?
3. What are the top 15 directories using more space?
47. vi/vim (text editor) (i)
vi text_file.txt (open "text_file.txt")
i - start edition mode (remember "insert")
ESC - stop edition mode
:w - save file ("write")
:q - quit
:x - save (write) and quit
48. vi/vim (text editor) (ii)
u - undo
:30 - go to line number 30
:syntax on - syntax highlighting
^ - go to beginning of line
$ - go to end of line
49. vi/vim (text editor) (iii)
dd - delete current line
d2↓ - delete current line and 2 lines below
yy - copy current line
y3↓ - copy current line and 3 lines below
pp - paste lines below current line
50. grep - finds words/patterns in a file (i)
grep word file.txt
Options:
grep -w : find the whole word
grep -c : returns the number of lines found
grep -f : specifies a file with a list of words
grep -o : returns only the match
51. grep - finds words/patterns in a file (ii)
grep -A 2 : also show 2 lines after
grep -B 3 : also show 3 lines before
grep -v : shows lines without pattern
grep --color : colors the match
52. Exercises
1. How many CNVs are located on chrom. 1?
2. How many deletions are there?
3. Which samples finish with character M?
4. Which samples finish with character M or F?
5. How many samples do not have NN in the
name?