HOW TO ACCESS THE CODE 

IN LARGE CODEBASE WITH VIM
Edgar Wang & Howard Chung & Ferry Chen
Titan Camera Team, 2014-2015
MILLION LINES OF CODE!!
External camera ISP use non-standard solution provided by QCOM
non-streaming output for still image capturing
External ISP wrapped asYUV sensor
wrappedYUV sensor not in QCOM expectations
YUV sensor with META!!!
REALLY?
MILLION LINES OF CODE!!
external ISP not provide CCI compatible commands
external ISP use SPI to download the firmware, SPI is not a
part of cameraV4L2 driver framework..
DOUBLE WHAMMY !!
MILLION LINES OF CODE!!
many features requested need the changes cross all layers from
APP to ISP, and the non-standard solution here make
THE WORST…
QCOM CANNOT support us with its PREPARATORY
code…
MILLION LINES OF CODE!!
HOW MUCH?
MILLION LINES OF CODE!!
PATH LOC
FRAMEWORKS/BASE/CORE/JAVA/ANDROID/HARDWARE/CAMERA.JAVA 1897
FRAMEWORK/BASE/CORE/JAVA/ANDROID/HARDWARE/CAMERA2/* 15099
FRAMEWORKS/BASE/CORE/JNI/ANDROID_HARDWARE_CAMERA.CPP 1021
FRAMEWORKS/BASE/CORE/JNI/ANDROID_HARDWARE_CAMERA2_* 2923
FRAMEWORKS/AV/SERVICES/CAMERA/LIBCAMERASERVICE/* 19940
FRAMEWORKS/AV/CAMERA/* 5492
HARDWARE/QCOM/CAMERA/* 103386
VENDOR/QCOM/PROPRIETARY/MM-CAMERA/* 816521
KERNEL/DRIVERS/MEDIA/PLATFORM/MSM/CAMERA_V2/* 51954
KERNEL/ARCH/ARM64/BOOT/DTS/QCOM/MSM8994-CAMERA.DTSI 517
KERNEL/ARCH/ARM64/BOOT/DTS/QCOM/MSM8994-TITAN-CAMERA.DTSI 0
SUM 1018750
1,000,000+
Lines of Code
(exclude the
comments)
MILLION LINES OF CODE!!
PATH #OFCHG
FRAMEWORKS/BASE/CORE/JAVA/ANDROID/HARDWARE/CAMERA.JAVA 114
FRAMEWORK/BASE/CORE/JAVA/ANDROID/HARDWARE/CAMERA2/* 1544
FRAMEWORKS/BASE/CORE/JNI/ANDROID_HARDWARE_CAMERA.CPP 4
FRAMEWORKS/BASE/CORE/JNI/ANDROID_HARDWARE_CAMERA2_* 153
FRAMEWORKS/AV/SERVICES/CAMERA/LIBCAMERASERVICE/* 329
FRAMEWORKS/AV/CAMERA/* 0
HARDWARE/QCOM/CAMERA/* 11059
VENDOR/QCOM/PROPRIETARY/MM-CAMERA/* 22120
KERNEL/DRIVERS/MEDIA/PLATFORM/MSM/CAMERA_V2/* 7271
KERNEL/ARCH/ARM64/BOOT/DTS/QCOM/MSM8994-CAMERA.DTSI 4
KERNEL/ARCH/ARM64/BOOT/DTS/QCOM/MSM8994-TITAN-CAMERA.DTSI 110
SUM 42708
4.2%,42078 linechanged by
Camera Team
CHALLENGES
Code Tracing
Features Implementation
Baseline Porting
Human Communication
Change submission
CHALLENGES - DETAILS
Call Path too long to trace
Feature’s implementation not only in 1 or 2
projects only
Porting - be the a real F! thing.
Hardly to talk with other guys in code level.
High missing rate with submission
Slow disk search
model complexities
etc……
DISLIKED
Exhausted code searching, this take lots of time to do it and degrade our performance!
e.g. grep, find, egrep…etc
Unnecessary directly switchings, redundant works and easy to lost in codebase
e.g. cd ../../../../../ && cd <path_to_hal>
No links between caller/callee for constants and functions
This will be a nightmare when tracing the call with tens of hops
Dupe and repeated works
This real be a hated one when you do thousands of changes!!
Slow deployment process
Flashing system takes 6 mins; push tons of libraries also take tens of seconds and error-prone.
Code editor only support one PL well
We are in Java/C++/C mixed codebase.
WANTED
Easy, Simple & one hop switching between working folders &
commands
Know the position of any symbol in a second
Trivial way to trace symbols with their caller and callee
Tabbed editor with FS navigator and list of symbols
The editor can support queries on symbols with tabs
The editor will be nice if it has bookmark abilities
THE START POINT…
Alfred Huang (ajch), early
camera BSP Owner
Switching directories really a
bullshit!
History commands not
straightforward to use
Vi is good to edit, but not
good to do code tracing
THE START POINT…
We must (by ajch)
more powerful cd command
reuse and recall command
history with ease
enableVi have full capability
to do code tracing
ESSENTIAL COMMANDS
ecd - extended cd
the memo list of working directories
support quick switch between working directories
ease of list modification
sorting on memo list
ESSENTIAL COMMANDS
Usage
For any working directory, using following command to push it into list
Whenever your want to jump a recorded directory, get the idx for jumping first
And you can jump to the directory by
~ $ ecd .
~ $ ecd -l
~ $ ecd -<Number>
ESSENTIAL COMMANDS
~ $ ecd -l
0) /home/cm/workspaces/Edgar/V06_1260/LINUX/android
1) /home/cm/workspaces/Edgar/V06_1260/LINUX/android/kernel/drivers/media/platform/msm/camera_v2
2) /home/cm/workspaces/Edgar/V06_1260/LINUX/android/vendor/qcom/proprietary/mm-camera-titan
3) /home/cm/workspaces/Edgar/V06_1260/LINUX/android/hardware/qcom/camera_titan
4) /home/cm/workspaces/Edgar/V06_1260/LINUX/android/vendor/qcom/proprietary/mm-still
5) /home/cm/workspaces/Edgar/V06_1260_55A/LINUX/android
6) /home/cm/workspaces/Edgar/V06_1260_55A/LINUX/android/hardware/qcom/camera_titan
7) /home/cm/workspaces/Edgar/V06_1260_55A/LINUX/android/vendor/qcom/proprietary
~ $ ecd -7
ESSENTIAL COMMANDS
hist - sort out cached history in shell and .bash_history
provide a visible view with command history in both memory
and .bash_history
not broken the bash’s command history but extend it
compatible to the exclamation mark (!) in bash
ESSENTIAL COMMANDS
Usage
For any working directory, using following command to push it into list
just pass the number after exclamation mark
~ $ hist|grep <keyword>
~ $ !<Number>
ESSENTIAL COMMANDS
~ $ hist|grep fastboot

220) adb shell logcat -v threadtime > ./log.txt
314) adb reboot bootloader && fastboot -i 0x27F1 flash boot /Volumes/Sanctuary/Ferry/V06_1260/LINUX/
android/out/target/product/v06/boot.img && fastboot -i 0x27F1 flash system /Volumes/Sanctuary/Ferry/
V06_1260/LINUX/android/out/target/product/v06/system.img && xfastflash reboot
317) adb shell setprop log.tag.ImageReaderTest VERBOSE
320) adb pull /storage/emulated/0/3840x2160_full_jpeg.jpg
321) adb pull /storage/emulated/0/3840x2160_full_yuv.jpg
368) adb shell logcat -v threadtime|grep "left ="
~ $ !314
WANTED
Easy, Simple & one hop switching between working folders &
commands
Know the position of any symbol in a second
Trivial way to trace symbols with their caller and callee
Tabbed editor with FS navigator and list of symbols
The editor can support queries on symbols with tabs
The editor will be nice if it has bookmark abilities
GTAGS - GNU GLOBAL source code tagging system
support 6 PLs by default, 25 PLs with Exuberant Ctags plug-in parser. Java/C/C++
are default ones.
Support shell command line,Vi, Emacs
Performance!
support text pattern
…etc
https://www.gnu.org/software/global/ for details…
GET SYMBOLS IN A SECOND
GET SYMBOLS IN A SECOND
Preparation
Download
Install gtags
make and install
~ $ wget http://tamacom.com/global/global-6.5.1.tar.gz
~ $ tar zxvf global-6.5.1.tar.gz && cd global-6.5.1
~ $ ./configure && make && mkaeinstall
GET SYMBOLS IN A SECOND
Use gtags for Android codebase
switch to Android source tree
generate list of file to be indexed (optional)
android_src $ find . ( ! -path "./out*" -a ! -name ".git*" ) -a ( -
name "*.cpp" -o -name "*.h" -o -name "*.c" -o -name "*.aidl" -o -
name "*.java" -o -name "*.S" ) > somefile
~ user$ cd <Android_source_tree>
GET SYMBOLS IN A SECOND
Index tags for Android codebase
Index the source tree
Update the source tree (when needed)
android_src $ gtags -f somefile
android_src $ global -uv
GET SYMBOLS IN A SECOND
Use gtags for indexed Android codebase
query a symbol
android_src $ global PowerManagerService
frameworks/base/services/core/java/com/android/server/power/PowerManagerService.java
GET SYMBOLS IN A SECOND
Use gtags for indexed Android codebase
query a symbol, have more info
android_src $ global -x PowerManagerService
PowerManagerService 102 ../../../../../frameworks/base/services/core/java/com/android/
server/power/PowerManagerService.java public final class PowerManagerService extends
SystemService
GET SYMBOLS IN A SECOND
Use gtags for indexed Android codebase
query WHO use a symbol
android_src $ global -r PowerManagerService
frameworks/base/services/core/java/com/android/server/power/PowerManagerService.java
frameworks/base/services/core/java/com/android/server/power/ShutdownThread.java
frameworks/base/services/java/com/android/server/SystemServer.java
GET SYMBOLS IN A SECOND
Use gtags for indexed Android codebase
query WHO use a symbol, with more info
android_src $ global -rx PowerManagerService
PowerManagerService 2998 frameworks/base/services/core/java/com/android/server/power/
PowerManagerService.java PowerManagerService.this.handleWakeLockDeath(this);
PowerManagerService 53 frameworks/base/services/core/java/com/android/server/power/
ShutdownThread.java import com.android.server.power.PowerManagerService;
PowerManagerService 622 frameworks/base/services/core/java/com/android/server/power/
ShutdownThread.java PowerManagerService.lowLevelReboot(reason);
PowerManagerService 644 frameworks/base/services/core/java/com/android/server/power/
ShutdownThread.java PowerManagerService.lowLevelShutdown();
…
GET SYMBOLS IN A SECOND
Use gtags for indexed Android codebase
query symbol support patterns (regular expression!)
android_src $ global PowerManager*
cts/tests/tests/os/src/android/os/cts/PowerManagerTest.java
cts/tests/tests/os/src/android/os/cts/PowerManager_WakeLockTest.java
…
Caution! This takes minutes!
But still much faster than search full
codebase..
WANTED
Easy, Simple & one hop switching between working folders &
commands
Know the position of any symbol in a second
Trivial way to trace symbols with their caller and callee
Tabbed editor with FS navigator and list of symbols
The editor can support queries on symbols with tabs
The editor will be nice if it has bookmark abilities
Vim/Vi - native supports on tabs
NerdTree - explore filesystem and to open files and directories forVi/Vim
Tagbar - list symbols of current file forVi/Vim
This depends on Exuberant ctags
TABBED EDITOR WITH FS
NAVIGATOR AND LIST OF SYMBOLS
android_src $ sudo apt-get install exuberant-ctags
Linux
android_src $ brew install exuberant-ctags
Mac
WANTED
Easy, Simple & one hop switching between working folders &
commands
Know the position of any symbol in a second
Trivial way to trace symbols with their caller and callee
Tabbed editor with FS navigator and list of symbols
The editor can support queries on symbols with tabs
The editor will be nice if it has bookmark abilities
GTAGS - GNU GLOBAL source code tagging system
Although gtags support much editors, but it is not good
enough!
Only jump to symbol(callee) supported by gtags…
TRIVIAL WAYTOTRACE SYMBOLS
TRIVIAL WAYTOTRACE SYMBOLS
ajch’s gtags.vim - Alfred’s gtags plugin formVi/Vim
we’re all vimers!!
support get the list callers of symbols!
the last pulse of code tracing!
TRIVIAL WAYTOTRACE SYMBOLS
TRIVIAL WAYTOTRACE SYMBOLS
Having list of caller and callee
is not enough
We must have next page/prev
page function too.
ajch’s mym.vim - stack behaviours of the gtags operations onVi/Vim
Hooray! we’re all vimers!!
support next page/prev page operations
makesVi/Vim be a browser like editor!
Alfred’s plugins are integrated to ourVimSuite - later on..
TRIVIAL WAYTOTRACE SYMBOLS
WANTED
Easy, Simple & one hop switching between working folders &
commands
Know the position of any symbol in a second
Trivial way to trace symbols with their caller and callee
Tabbed editor with FS navigator and list of symbols
The editor can support queries on symbols with tabs
The editor will be nice if it has bookmark abilities
QFEnter - A vim plugin for intuitive file opening from Quickfix
window to tabs
THE EDITOR CAN SUPPORT
QUERIES ON SYMBOLS WITHTABS
vim-bookmarks - A vim plugin for bookmark
support line based bookmark
support bookmark annotations
ease of use
integrated to quick fix window
…
THE EDITOR WILL BE NICE IF IT
HAS BOOKMARK ABILITIES
EHFANDROIDVIMSUITE
F2 - open/close/update bookmark
F7
F8
F5 - next tab
F6 - prev tab
t
EHFANDROIDVIMSUITE
F3 - list of defines
F4 - list of references
WANTED
Easy, Simple & one hop switching between working folders &
commands
Know the position of any symbol in a second
Trivial way to trace symbols with their caller and callee
Tabbed editor with FS navigator and list of symbols
The editor can support queries on symbols with tabs
The editor will be nice if it has bookmark abilities
It is not enough
It can be better
WANTED II
Random access to codebase
Not only know the position of symbol, but put editor to the line
directly
Dupe and Repeated works can be maximum eliminated
Deployment have quick, accurate, and easy solution
A secure and safe way to backup/save our works with trials
More & more…
xvim - put you to the symbol withVi/Vim directly
based on gtags.
ease of use
list all candidates to choose first
great partner to debug with logs!!!!
RANDOM ACCESSTO CODEBASE
RANDOM ACCESSTO CODEBASE
10-09 08:46:54.239 7427 7427 E mm-camera: mct: mct_pipeline_stop_session - E
10-09 08:46:54.240 7427 7427 E mm-camera: mct:
mct_pipeline_process_set:command=800000b
10-09 08:46:54.240 7427 7427 E mm-camera: mct: mct_pipeline_process_set: Issuing
DEL_STREAM on stream 0x1000f and stream type=10
10-09 08:46:54.240 7427 7427 I mm-camera-sensor: port_sensor_unlink_func:293ide 1000f
10-09 08:46:54.240 7427 7427 E mm-camera: port_iface_ext_unlink_func: E, identity =
0x1000f, port = 0xf4878e20, direction = 2
10-09 08:46:54.240 7427 7427 I mm-camera-sensor: port_sensor_caps_unreserve:204ide 1000f
10-09 08:46:54.240 7427 7427 E mm-camera: port_iface_check_caps_unreserve_func: E,
identity = 0x1000f, port = 0xf4878e20, direction = 2
10-09 08:46:54.240 7427 7427 E mm-camera: iface_unreserve_sink_port: stream_id f mode 0
10-09 08:46:54.240 7427 7427 E mm-camera: port_iface_ext_unlink_func: E, identity =
0x1000f, port = 0xf4bbc160, direction = 1
10-09 08:46:54.240 7427 7427 E mm-camera: iface_ext_unlink_src_port: mct stream already
removed from hw stream
RANDOM ACCESSTO CODEBASE
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:787)
at java.util.HashMap$EntryIterator.next(HashMap.java:824)
at java.util.HashMap$EntryIterator.next(HashMap.java:822)
at com.android.internal.telephony.SubscriptionController.getPhoneId(SubscriptionController.java:1199)
at com.android.phone.PhoneInterfaceManager.getPhone(PhoneInterfaceManager.java:789)
at com.android.phone.PhoneInterfaceManager.getIccId(PhoneInterfaceManager.java:2161)
at com.android.phone.PhoneInterfaceManager.getLine1NumberForDisplay(PhoneInterfaceManager.java:2205)
…
drop you to the symbol directly!
RANDOM ACCESSTO CODEBASE
android_src $ xvim <symbol_name>
android_src $ xvim port_iface_check_caps_unreserve_func
Example
android_src $ xvim getPhoneId
RANDOM ACCESSTO CODEBASE
android_src $ xvim getPhoneId

1) frameworks/base/packages/SystemUI/src/com/android/systemui/statusbar/MSimSignalClusterView.java:
378
2) frameworks/base/packages/SystemUI/src/com/android/systemui/statusbar/policy/
MSimNetworkControllerImpl.java:280
3) frameworks/base/telephony/java/android/telephony/RadioAccessFamily.java:75
4) frameworks/base/telephony/java/android/telephony/SubscriptionManager.java:828
5) frameworks/opt/telephony/src/java/com/android/internal/telephony/Phone.java:1999
6) frameworks/opt/telephony/src/java/com/android/internal/telephony/PhoneBase.java:2203
7) frameworks/opt/telephony/src/java/com/android/internal/telephony/PhoneProxy.java:1470
8) frameworks/opt/telephony/src/java/com/android/internal/telephony/ServiceStateTracker.java:1101
9) frameworks/opt/telephony/src/java/com/android/internal/telephony/SubscriptionController.java:1169
…
#?
WANTED II
Random access to codebase
Not only know the position of symbol, but put editor to the line
directly
Dupe and Repeated works can be maximum eliminated
Deployment have quick, accurate, and easy solution
A secure and safe way to backup/save our works with trials
More & more…
The problem…
comments for jira header/footer really a bullshit when code
changes in 4+ files…
comments for jira header/footer, this is penalty to RD, it
discourage RD to do changes to solve the problem!
DUPE AND REPEATED WORKS
CAN BE MAXIMUM ELIMINATED
git xcommit - unlock you from jira comment blocks..
auto comment jira header/fooder surround changed code block
interactive way to input jira info for git-commits.
auto list changed files for git add and commit msg.
DUPE AND REPEATED WORKS
CAN BE MAXIMUM ELIMINATED
PREFERRED II
Random access to codebase
Not only know the position of symbol, but put editor to the line
directly
Dupe and Repeated works can be maximum eliminated
Deployment have quick, accurate, and easy solution
A secure and safe way to backup/save our works with trials
More & more…
The problem….
DEPLOYMENT HAVE QUICK,
ACCURATE,AND EASY SOLUTION
project_XX $ mm
project_XX $ cd ../../../../../../out/target/prodcut/V99
V99 $ adb push system/…. /system/….
V99 $ cd -
project_XX $ mm
reinstallSomething.py
auto delivery .apk, .so, .xml, executable binaries to device with
give name
support pattern search
accurate, reliable, and support both local and remote repositories
DEPLOYMENT HAVE QUICK,
ACCURATE,AND EASY SOLUTION
DEPLOYMENT HAVE QUICK,
ACCURATE,AND EASY SOLUTION
project_XX $ mm
…
Install: out/target/product/v06/system/lib/hw/camera.msm8994.so
…
Install: out/target/product/v06/system/lib/libmmcamera_interface.so
…
Install: out/target/product/v06/system/lib/libmm-qcamera.so
…..
…..
project_XX $ adb root
project_XX $ adb remount
project_XX $ reinstallSomething.py -i camera.msm8994.so libmmcamera_interface.so libmm-qcamera.so
project_XX$ adb reboot
pattern support
Support remote repository
DEPLOYMENT HAVE QUICK,
ACCURATE,AND EASY SOLUTION
project_XX $ reinstallSomething.py -i libmmcamera*
local_dir $ export OUT_OVERLAY=/Volumes/Sanctuary/Ferry/V06_1260/LINUX/android/out/target/
product/v06
local_dir $ reinstallSomething.py -I libmmcamera*
local_dir $ adb reboot
WANTED II
Random access to codebase
Not only know the position of symbol, but put editor to the line
directly
Dupe and Repeated works can be maximum eliminated
Deployment have quick, accurate, and easy solution
A secure and safe way to backup/save our works with trials
More & more…
Problem..
the patch is good… but sometimes not applicable
how to have full picture of changes
how to deliver the change to other guys with full contexts
…
A SECURE AND SAFE WAYTO BACKUP/
SAVE OUR WORKS WITHTRIALS
git xcommit -backup
This command will backup modified things (note, new added one must be
separately handled, doesn’t a matter)
A SECURE AND SAFE WAYTO BACKUP/
SAVE OUR WORKS WITHTRIALS
folder_to_backup $ git xcommit -backup
A SECURE AND SAFE WAYTO BACKUP/
SAVE OUR WORKS WITHTRIALS
PREFERRED II
Random access to codebase
Not only know the position of symbol, but put editor to the line
directly
Dupe and Repeated works can be maximum eliminated
Deployment have quick, accurate, and easy solution
A secure and safe way to backup/save our works with trials
More & more…
xfastboot, xadb…
when multiple connected to host……
MORE & MORE…
user $ xadb shell
more than 1 devices, choose 1 device:
1) V06001325
2) V06000136
3) V06001835
4) V06000137
#? 1
V06001325
root@v06:/ #
LINKS
https://github.com/ferrisky/EHFAndroidVimSuite
https://github.com/ferrisky/ecd
https://github.com/ferrisky/hist
https://github.com/xygod/Git_X_Commit
https://github.com/xygod/Gtags_X_Vim
https://github.com/xygod/reinstallSomething
WELCOME TO CONTRIBUTE
CODES TO US!

How To Access Code In Large w/ Vim

  • 1.
    HOW TO ACCESSTHE CODE 
 IN LARGE CODEBASE WITH VIM Edgar Wang & Howard Chung & Ferry Chen Titan Camera Team, 2014-2015
  • 2.
    MILLION LINES OFCODE!! External camera ISP use non-standard solution provided by QCOM non-streaming output for still image capturing External ISP wrapped asYUV sensor wrappedYUV sensor not in QCOM expectations YUV sensor with META!!! REALLY?
  • 3.
    MILLION LINES OFCODE!! external ISP not provide CCI compatible commands external ISP use SPI to download the firmware, SPI is not a part of cameraV4L2 driver framework.. DOUBLE WHAMMY !!
  • 4.
    MILLION LINES OFCODE!! many features requested need the changes cross all layers from APP to ISP, and the non-standard solution here make THE WORST… QCOM CANNOT support us with its PREPARATORY code…
  • 5.
    MILLION LINES OFCODE!! HOW MUCH?
  • 6.
    MILLION LINES OFCODE!! PATH LOC FRAMEWORKS/BASE/CORE/JAVA/ANDROID/HARDWARE/CAMERA.JAVA 1897 FRAMEWORK/BASE/CORE/JAVA/ANDROID/HARDWARE/CAMERA2/* 15099 FRAMEWORKS/BASE/CORE/JNI/ANDROID_HARDWARE_CAMERA.CPP 1021 FRAMEWORKS/BASE/CORE/JNI/ANDROID_HARDWARE_CAMERA2_* 2923 FRAMEWORKS/AV/SERVICES/CAMERA/LIBCAMERASERVICE/* 19940 FRAMEWORKS/AV/CAMERA/* 5492 HARDWARE/QCOM/CAMERA/* 103386 VENDOR/QCOM/PROPRIETARY/MM-CAMERA/* 816521 KERNEL/DRIVERS/MEDIA/PLATFORM/MSM/CAMERA_V2/* 51954 KERNEL/ARCH/ARM64/BOOT/DTS/QCOM/MSM8994-CAMERA.DTSI 517 KERNEL/ARCH/ARM64/BOOT/DTS/QCOM/MSM8994-TITAN-CAMERA.DTSI 0 SUM 1018750 1,000,000+ Lines of Code (exclude the comments)
  • 7.
    MILLION LINES OFCODE!! PATH #OFCHG FRAMEWORKS/BASE/CORE/JAVA/ANDROID/HARDWARE/CAMERA.JAVA 114 FRAMEWORK/BASE/CORE/JAVA/ANDROID/HARDWARE/CAMERA2/* 1544 FRAMEWORKS/BASE/CORE/JNI/ANDROID_HARDWARE_CAMERA.CPP 4 FRAMEWORKS/BASE/CORE/JNI/ANDROID_HARDWARE_CAMERA2_* 153 FRAMEWORKS/AV/SERVICES/CAMERA/LIBCAMERASERVICE/* 329 FRAMEWORKS/AV/CAMERA/* 0 HARDWARE/QCOM/CAMERA/* 11059 VENDOR/QCOM/PROPRIETARY/MM-CAMERA/* 22120 KERNEL/DRIVERS/MEDIA/PLATFORM/MSM/CAMERA_V2/* 7271 KERNEL/ARCH/ARM64/BOOT/DTS/QCOM/MSM8994-CAMERA.DTSI 4 KERNEL/ARCH/ARM64/BOOT/DTS/QCOM/MSM8994-TITAN-CAMERA.DTSI 110 SUM 42708 4.2%,42078 linechanged by Camera Team
  • 8.
    CHALLENGES Code Tracing Features Implementation BaselinePorting Human Communication Change submission
  • 9.
    CHALLENGES - DETAILS CallPath too long to trace Feature’s implementation not only in 1 or 2 projects only Porting - be the a real F! thing. Hardly to talk with other guys in code level. High missing rate with submission Slow disk search model complexities etc……
  • 10.
    DISLIKED Exhausted code searching,this take lots of time to do it and degrade our performance! e.g. grep, find, egrep…etc Unnecessary directly switchings, redundant works and easy to lost in codebase e.g. cd ../../../../../ && cd <path_to_hal> No links between caller/callee for constants and functions This will be a nightmare when tracing the call with tens of hops Dupe and repeated works This real be a hated one when you do thousands of changes!! Slow deployment process Flashing system takes 6 mins; push tons of libraries also take tens of seconds and error-prone. Code editor only support one PL well We are in Java/C++/C mixed codebase.
  • 11.
    WANTED Easy, Simple &one hop switching between working folders & commands Know the position of any symbol in a second Trivial way to trace symbols with their caller and callee Tabbed editor with FS navigator and list of symbols The editor can support queries on symbols with tabs The editor will be nice if it has bookmark abilities
  • 12.
    THE START POINT… AlfredHuang (ajch), early camera BSP Owner Switching directories really a bullshit! History commands not straightforward to use Vi is good to edit, but not good to do code tracing
  • 13.
    THE START POINT… Wemust (by ajch) more powerful cd command reuse and recall command history with ease enableVi have full capability to do code tracing
  • 14.
    ESSENTIAL COMMANDS ecd -extended cd the memo list of working directories support quick switch between working directories ease of list modification sorting on memo list
  • 15.
    ESSENTIAL COMMANDS Usage For anyworking directory, using following command to push it into list Whenever your want to jump a recorded directory, get the idx for jumping first And you can jump to the directory by ~ $ ecd . ~ $ ecd -l ~ $ ecd -<Number>
  • 16.
    ESSENTIAL COMMANDS ~ $ecd -l 0) /home/cm/workspaces/Edgar/V06_1260/LINUX/android 1) /home/cm/workspaces/Edgar/V06_1260/LINUX/android/kernel/drivers/media/platform/msm/camera_v2 2) /home/cm/workspaces/Edgar/V06_1260/LINUX/android/vendor/qcom/proprietary/mm-camera-titan 3) /home/cm/workspaces/Edgar/V06_1260/LINUX/android/hardware/qcom/camera_titan 4) /home/cm/workspaces/Edgar/V06_1260/LINUX/android/vendor/qcom/proprietary/mm-still 5) /home/cm/workspaces/Edgar/V06_1260_55A/LINUX/android 6) /home/cm/workspaces/Edgar/V06_1260_55A/LINUX/android/hardware/qcom/camera_titan 7) /home/cm/workspaces/Edgar/V06_1260_55A/LINUX/android/vendor/qcom/proprietary ~ $ ecd -7
  • 17.
    ESSENTIAL COMMANDS hist -sort out cached history in shell and .bash_history provide a visible view with command history in both memory and .bash_history not broken the bash’s command history but extend it compatible to the exclamation mark (!) in bash
  • 18.
    ESSENTIAL COMMANDS Usage For anyworking directory, using following command to push it into list just pass the number after exclamation mark ~ $ hist|grep <keyword> ~ $ !<Number>
  • 19.
    ESSENTIAL COMMANDS ~ $hist|grep fastboot
 220) adb shell logcat -v threadtime > ./log.txt 314) adb reboot bootloader && fastboot -i 0x27F1 flash boot /Volumes/Sanctuary/Ferry/V06_1260/LINUX/ android/out/target/product/v06/boot.img && fastboot -i 0x27F1 flash system /Volumes/Sanctuary/Ferry/ V06_1260/LINUX/android/out/target/product/v06/system.img && xfastflash reboot 317) adb shell setprop log.tag.ImageReaderTest VERBOSE 320) adb pull /storage/emulated/0/3840x2160_full_jpeg.jpg 321) adb pull /storage/emulated/0/3840x2160_full_yuv.jpg 368) adb shell logcat -v threadtime|grep "left =" ~ $ !314
  • 20.
    WANTED Easy, Simple &one hop switching between working folders & commands Know the position of any symbol in a second Trivial way to trace symbols with their caller and callee Tabbed editor with FS navigator and list of symbols The editor can support queries on symbols with tabs The editor will be nice if it has bookmark abilities
  • 21.
    GTAGS - GNUGLOBAL source code tagging system support 6 PLs by default, 25 PLs with Exuberant Ctags plug-in parser. Java/C/C++ are default ones. Support shell command line,Vi, Emacs Performance! support text pattern …etc https://www.gnu.org/software/global/ for details… GET SYMBOLS IN A SECOND
  • 22.
    GET SYMBOLS INA SECOND Preparation Download Install gtags make and install ~ $ wget http://tamacom.com/global/global-6.5.1.tar.gz ~ $ tar zxvf global-6.5.1.tar.gz && cd global-6.5.1 ~ $ ./configure && make && mkaeinstall
  • 23.
    GET SYMBOLS INA SECOND Use gtags for Android codebase switch to Android source tree generate list of file to be indexed (optional) android_src $ find . ( ! -path "./out*" -a ! -name ".git*" ) -a ( - name "*.cpp" -o -name "*.h" -o -name "*.c" -o -name "*.aidl" -o - name "*.java" -o -name "*.S" ) > somefile ~ user$ cd <Android_source_tree>
  • 24.
    GET SYMBOLS INA SECOND Index tags for Android codebase Index the source tree Update the source tree (when needed) android_src $ gtags -f somefile android_src $ global -uv
  • 25.
    GET SYMBOLS INA SECOND Use gtags for indexed Android codebase query a symbol android_src $ global PowerManagerService frameworks/base/services/core/java/com/android/server/power/PowerManagerService.java
  • 26.
    GET SYMBOLS INA SECOND Use gtags for indexed Android codebase query a symbol, have more info android_src $ global -x PowerManagerService PowerManagerService 102 ../../../../../frameworks/base/services/core/java/com/android/ server/power/PowerManagerService.java public final class PowerManagerService extends SystemService
  • 27.
    GET SYMBOLS INA SECOND Use gtags for indexed Android codebase query WHO use a symbol android_src $ global -r PowerManagerService frameworks/base/services/core/java/com/android/server/power/PowerManagerService.java frameworks/base/services/core/java/com/android/server/power/ShutdownThread.java frameworks/base/services/java/com/android/server/SystemServer.java
  • 28.
    GET SYMBOLS INA SECOND Use gtags for indexed Android codebase query WHO use a symbol, with more info android_src $ global -rx PowerManagerService PowerManagerService 2998 frameworks/base/services/core/java/com/android/server/power/ PowerManagerService.java PowerManagerService.this.handleWakeLockDeath(this); PowerManagerService 53 frameworks/base/services/core/java/com/android/server/power/ ShutdownThread.java import com.android.server.power.PowerManagerService; PowerManagerService 622 frameworks/base/services/core/java/com/android/server/power/ ShutdownThread.java PowerManagerService.lowLevelReboot(reason); PowerManagerService 644 frameworks/base/services/core/java/com/android/server/power/ ShutdownThread.java PowerManagerService.lowLevelShutdown(); …
  • 29.
    GET SYMBOLS INA SECOND Use gtags for indexed Android codebase query symbol support patterns (regular expression!) android_src $ global PowerManager* cts/tests/tests/os/src/android/os/cts/PowerManagerTest.java cts/tests/tests/os/src/android/os/cts/PowerManager_WakeLockTest.java … Caution! This takes minutes! But still much faster than search full codebase..
  • 30.
    WANTED Easy, Simple &one hop switching between working folders & commands Know the position of any symbol in a second Trivial way to trace symbols with their caller and callee Tabbed editor with FS navigator and list of symbols The editor can support queries on symbols with tabs The editor will be nice if it has bookmark abilities
  • 31.
    Vim/Vi - nativesupports on tabs NerdTree - explore filesystem and to open files and directories forVi/Vim Tagbar - list symbols of current file forVi/Vim This depends on Exuberant ctags TABBED EDITOR WITH FS NAVIGATOR AND LIST OF SYMBOLS android_src $ sudo apt-get install exuberant-ctags Linux android_src $ brew install exuberant-ctags Mac
  • 32.
    WANTED Easy, Simple &one hop switching between working folders & commands Know the position of any symbol in a second Trivial way to trace symbols with their caller and callee Tabbed editor with FS navigator and list of symbols The editor can support queries on symbols with tabs The editor will be nice if it has bookmark abilities
  • 33.
    GTAGS - GNUGLOBAL source code tagging system Although gtags support much editors, but it is not good enough! Only jump to symbol(callee) supported by gtags… TRIVIAL WAYTOTRACE SYMBOLS
  • 34.
  • 35.
    ajch’s gtags.vim -Alfred’s gtags plugin formVi/Vim we’re all vimers!! support get the list callers of symbols! the last pulse of code tracing! TRIVIAL WAYTOTRACE SYMBOLS
  • 36.
    TRIVIAL WAYTOTRACE SYMBOLS Havinglist of caller and callee is not enough We must have next page/prev page function too.
  • 37.
    ajch’s mym.vim -stack behaviours of the gtags operations onVi/Vim Hooray! we’re all vimers!! support next page/prev page operations makesVi/Vim be a browser like editor! Alfred’s plugins are integrated to ourVimSuite - later on.. TRIVIAL WAYTOTRACE SYMBOLS
  • 38.
    WANTED Easy, Simple &one hop switching between working folders & commands Know the position of any symbol in a second Trivial way to trace symbols with their caller and callee Tabbed editor with FS navigator and list of symbols The editor can support queries on symbols with tabs The editor will be nice if it has bookmark abilities
  • 39.
    QFEnter - Avim plugin for intuitive file opening from Quickfix window to tabs THE EDITOR CAN SUPPORT QUERIES ON SYMBOLS WITHTABS
  • 40.
    vim-bookmarks - Avim plugin for bookmark support line based bookmark support bookmark annotations ease of use integrated to quick fix window … THE EDITOR WILL BE NICE IF IT HAS BOOKMARK ABILITIES
  • 41.
    EHFANDROIDVIMSUITE F2 - open/close/updatebookmark F7 F8 F5 - next tab F6 - prev tab t
  • 42.
    EHFANDROIDVIMSUITE F3 - listof defines F4 - list of references
  • 43.
    WANTED Easy, Simple &one hop switching between working folders & commands Know the position of any symbol in a second Trivial way to trace symbols with their caller and callee Tabbed editor with FS navigator and list of symbols The editor can support queries on symbols with tabs The editor will be nice if it has bookmark abilities
  • 44.
    It is notenough It can be better
  • 45.
    WANTED II Random accessto codebase Not only know the position of symbol, but put editor to the line directly Dupe and Repeated works can be maximum eliminated Deployment have quick, accurate, and easy solution A secure and safe way to backup/save our works with trials More & more…
  • 46.
    xvim - putyou to the symbol withVi/Vim directly based on gtags. ease of use list all candidates to choose first great partner to debug with logs!!!! RANDOM ACCESSTO CODEBASE
  • 47.
    RANDOM ACCESSTO CODEBASE 10-0908:46:54.239 7427 7427 E mm-camera: mct: mct_pipeline_stop_session - E 10-09 08:46:54.240 7427 7427 E mm-camera: mct: mct_pipeline_process_set:command=800000b 10-09 08:46:54.240 7427 7427 E mm-camera: mct: mct_pipeline_process_set: Issuing DEL_STREAM on stream 0x1000f and stream type=10 10-09 08:46:54.240 7427 7427 I mm-camera-sensor: port_sensor_unlink_func:293ide 1000f 10-09 08:46:54.240 7427 7427 E mm-camera: port_iface_ext_unlink_func: E, identity = 0x1000f, port = 0xf4878e20, direction = 2 10-09 08:46:54.240 7427 7427 I mm-camera-sensor: port_sensor_caps_unreserve:204ide 1000f 10-09 08:46:54.240 7427 7427 E mm-camera: port_iface_check_caps_unreserve_func: E, identity = 0x1000f, port = 0xf4878e20, direction = 2 10-09 08:46:54.240 7427 7427 E mm-camera: iface_unreserve_sink_port: stream_id f mode 0 10-09 08:46:54.240 7427 7427 E mm-camera: port_iface_ext_unlink_func: E, identity = 0x1000f, port = 0xf4bbc160, direction = 1 10-09 08:46:54.240 7427 7427 E mm-camera: iface_ext_unlink_src_port: mct stream already removed from hw stream
  • 48.
    RANDOM ACCESSTO CODEBASE java.util.ConcurrentModificationException atjava.util.HashMap$HashIterator.nextEntry(HashMap.java:787) at java.util.HashMap$EntryIterator.next(HashMap.java:824) at java.util.HashMap$EntryIterator.next(HashMap.java:822) at com.android.internal.telephony.SubscriptionController.getPhoneId(SubscriptionController.java:1199) at com.android.phone.PhoneInterfaceManager.getPhone(PhoneInterfaceManager.java:789) at com.android.phone.PhoneInterfaceManager.getIccId(PhoneInterfaceManager.java:2161) at com.android.phone.PhoneInterfaceManager.getLine1NumberForDisplay(PhoneInterfaceManager.java:2205) …
  • 49.
    drop you tothe symbol directly! RANDOM ACCESSTO CODEBASE android_src $ xvim <symbol_name> android_src $ xvim port_iface_check_caps_unreserve_func Example android_src $ xvim getPhoneId
  • 50.
    RANDOM ACCESSTO CODEBASE android_src$ xvim getPhoneId
 1) frameworks/base/packages/SystemUI/src/com/android/systemui/statusbar/MSimSignalClusterView.java: 378 2) frameworks/base/packages/SystemUI/src/com/android/systemui/statusbar/policy/ MSimNetworkControllerImpl.java:280 3) frameworks/base/telephony/java/android/telephony/RadioAccessFamily.java:75 4) frameworks/base/telephony/java/android/telephony/SubscriptionManager.java:828 5) frameworks/opt/telephony/src/java/com/android/internal/telephony/Phone.java:1999 6) frameworks/opt/telephony/src/java/com/android/internal/telephony/PhoneBase.java:2203 7) frameworks/opt/telephony/src/java/com/android/internal/telephony/PhoneProxy.java:1470 8) frameworks/opt/telephony/src/java/com/android/internal/telephony/ServiceStateTracker.java:1101 9) frameworks/opt/telephony/src/java/com/android/internal/telephony/SubscriptionController.java:1169 … #?
  • 51.
    WANTED II Random accessto codebase Not only know the position of symbol, but put editor to the line directly Dupe and Repeated works can be maximum eliminated Deployment have quick, accurate, and easy solution A secure and safe way to backup/save our works with trials More & more…
  • 52.
    The problem… comments forjira header/footer really a bullshit when code changes in 4+ files… comments for jira header/footer, this is penalty to RD, it discourage RD to do changes to solve the problem! DUPE AND REPEATED WORKS CAN BE MAXIMUM ELIMINATED
  • 53.
    git xcommit -unlock you from jira comment blocks.. auto comment jira header/fooder surround changed code block interactive way to input jira info for git-commits. auto list changed files for git add and commit msg. DUPE AND REPEATED WORKS CAN BE MAXIMUM ELIMINATED
  • 54.
    PREFERRED II Random accessto codebase Not only know the position of symbol, but put editor to the line directly Dupe and Repeated works can be maximum eliminated Deployment have quick, accurate, and easy solution A secure and safe way to backup/save our works with trials More & more…
  • 55.
    The problem…. DEPLOYMENT HAVEQUICK, ACCURATE,AND EASY SOLUTION project_XX $ mm project_XX $ cd ../../../../../../out/target/prodcut/V99 V99 $ adb push system/…. /system/…. V99 $ cd - project_XX $ mm
  • 56.
    reinstallSomething.py auto delivery .apk,.so, .xml, executable binaries to device with give name support pattern search accurate, reliable, and support both local and remote repositories DEPLOYMENT HAVE QUICK, ACCURATE,AND EASY SOLUTION
  • 57.
    DEPLOYMENT HAVE QUICK, ACCURATE,ANDEASY SOLUTION project_XX $ mm … Install: out/target/product/v06/system/lib/hw/camera.msm8994.so … Install: out/target/product/v06/system/lib/libmmcamera_interface.so … Install: out/target/product/v06/system/lib/libmm-qcamera.so ….. ….. project_XX $ adb root project_XX $ adb remount project_XX $ reinstallSomething.py -i camera.msm8994.so libmmcamera_interface.so libmm-qcamera.so project_XX$ adb reboot
  • 58.
    pattern support Support remoterepository DEPLOYMENT HAVE QUICK, ACCURATE,AND EASY SOLUTION project_XX $ reinstallSomething.py -i libmmcamera* local_dir $ export OUT_OVERLAY=/Volumes/Sanctuary/Ferry/V06_1260/LINUX/android/out/target/ product/v06 local_dir $ reinstallSomething.py -I libmmcamera* local_dir $ adb reboot
  • 59.
    WANTED II Random accessto codebase Not only know the position of symbol, but put editor to the line directly Dupe and Repeated works can be maximum eliminated Deployment have quick, accurate, and easy solution A secure and safe way to backup/save our works with trials More & more…
  • 60.
    Problem.. the patch isgood… but sometimes not applicable how to have full picture of changes how to deliver the change to other guys with full contexts … A SECURE AND SAFE WAYTO BACKUP/ SAVE OUR WORKS WITHTRIALS
  • 61.
    git xcommit -backup Thiscommand will backup modified things (note, new added one must be separately handled, doesn’t a matter) A SECURE AND SAFE WAYTO BACKUP/ SAVE OUR WORKS WITHTRIALS folder_to_backup $ git xcommit -backup
  • 62.
    A SECURE ANDSAFE WAYTO BACKUP/ SAVE OUR WORKS WITHTRIALS
  • 63.
    PREFERRED II Random accessto codebase Not only know the position of symbol, but put editor to the line directly Dupe and Repeated works can be maximum eliminated Deployment have quick, accurate, and easy solution A secure and safe way to backup/save our works with trials More & more…
  • 64.
    xfastboot, xadb… when multipleconnected to host…… MORE & MORE… user $ xadb shell more than 1 devices, choose 1 device: 1) V06001325 2) V06000136 3) V06001835 4) V06000137 #? 1 V06001325 root@v06:/ #
  • 65.
  • 66.