6. AppChecker – совместимость с дистрибутивами of 20
7.
Editor's Notes
Shallow tests – simple tests with the only guaranteed purpose of ensuring the interface does not crash (in many cases it is additionally checked that the interface doesn’t return an error code) being called with some particular correct parameters and in the correct environment. This is close to “existence” or “smoke” or “sanity” tests (but beware - these terms are interpreted differently by different experts). We use a new technology and tools ( AZOV Framework ) for automatic shallow tests generation based on some description of the interfaces, their parameters, interlinks and default values in a database or based on header sources. The core idea here is to augment the database to contain enough information about the interfaces and their dependencies that would allow automatically building correct call chains representing typical scenarios of interface usage. Normal tests – this is the most reasonable level of testing quality. Normal tests check the main functionality and may check a few error scenarios. Most of the existing tests in the industry are of this quality. We have developed T2C Framework to help to automate a lot of tedious tasks in development of conformance tests allowing test developers to focus on test logic not on numerous auxiliary general issues. Drawing an analogy with programming languages, T2C enables effective writing tests in a “high level programming language” compared to “assembly” in manual test development. Deep tests – this is the level when most of the specification assertions are tested in various conditions/states. This level of testing is hard to achieve by manual tests - that’s why we have developed an advanced testing technology UniTESK to automate this. UniTESK is an model-driven technology for automated test development. The key point of the technology is automatic generation of tests based on formal requirement specifications and corresponding test scenarios. UniTESK gives the following benefits (though requiring very good engineers to be involved): Thorough testing is achieved relatively easy by making sophisticated computer algorithms responsible for test sequences generation based on high level scenario descriptions in SeC language. Separating specifications and test scenarios allows independently changing the tests as requirements evolve and improve test coverage thoroughness as resources appear. Automatic test generation allows smooth parameterization and customization thus facilitating adaptation of the tests for various usage conditions (e.g. for pared-down embedded Linux systems or extended enterprise requirements).
There are three major areas in which ISP RAS participates in collaboration with LF: The first thing is extending test coverage of the LSB tests. The problem is that now only about 20% of interfaces are somehow tested. We definitely need to develop more tests to cover 100% of interfaces. The quality of testing should depend on the importance of interfaces - deep quality for important interfaces, shallow testing for not important. The next thing is that we need to review and improve the quality of the standard text to make it suitable for using by application and upstream developers. We started these two activities in the OLVER project for LSB Core (1 500 interfaces) and need to extend our experience to the whole LSB (30 000 interfaces now). Finally we need to create new infrastructure for making LSB practically useful for the masses of application developers and allow LSB workgroup and its stakeholders easier and more consistently tracking the Linux evolution and reflect the industry needs in future versions of the standard. Currently we are developing the following things: The backbone thing is a central LSB database that need to hold all the LSB related information: LSB constituent elements, Linux ecosystem tracker (to track applications and distributions and their evolution in terms of compatibility), various information for LSB development decision making. LSB Navigator – a web portal for interaction between Linux developers, Linux distribution vendors and LSB workgroup. It will be an interactive system for querying, analyzing and submitting various information. The beta version is available at the linuxtesting site, the link is announced in the press articles (see the LF site). Test Execution Framework – a framework for user friendly execution of LSB tests and analysis of the results. The first version is already deployed in the production as announced in the newest Linux Foundation press release as a part of LSB 3.1 Update 1. Certification System – an automated framework for conducting distribution and application certification for LSB. All the systems must be integrated with information about all the key elements linked to each other - distributions, applications, upstream components, tests and LSB specification. Важное замечание – это текущий взгляд на задачи, но на самом деле постоянно появляются новые задачи и потребности.
Shallow tests – simple tests with the only guaranteed purpose of ensuring the interface does not crash (in many cases it is additionally checked that the interface doesn’t return an error code) being called with some particular correct parameters and in the correct environment. This is close to “existence” or “smoke” or “sanity” tests (but beware - these terms are interpreted differently by different experts). We use a new technology and tools ( AZOV Framework ) for automatic shallow tests generation based on some description of the interfaces, their parameters, interlinks and default values in a database or based on header sources. The core idea here is to augment the database to contain enough information about the interfaces and their dependencies that would allow automatically building correct call chains representing typical scenarios of interface usage. Normal tests – this is the most reasonable level of testing quality. Normal tests check the main functionality and may check a few error scenarios. Most of the existing tests in the industry are of this quality. We have developed T2C Framework to help to automate a lot of tedious tasks in development of conformance tests allowing test developers to focus on test logic not on numerous auxiliary general issues. Drawing an analogy with programming languages, T2C enables effective writing tests in a “high level programming language” compared to “assembly” in manual test development. Deep tests – this is the level when most of the specification assertions are tested in various conditions/states. This level of testing is hard to achieve by manual tests - that’s why we have developed an advanced testing technology UniTESK to automate this. UniTESK is an model-driven technology for automated test development. The key point of the technology is automatic generation of tests based on formal requirement specifications and corresponding test scenarios. UniTESK gives the following benefits (though requiring very good engineers to be involved): Thorough testing is achieved relatively easy by making sophisticated computer algorithms responsible for test sequences generation based on high level scenario descriptions in SeC language. Separating specifications and test scenarios allows independently changing the tests as requirements evolve and improve test coverage thoroughness as resources appear. Automatic test generation allows smooth parameterization and customization thus facilitating adaptation of the tests for various usage conditions (e.g. for pared-down embedded Linux systems or extended enterprise requirements).
![Linux Standard Base ,[object Object],[object Object],[object Object],[object Object],SE Department and LVC of 18](https://image.slidesharecdn.com/lsb-slides-110325062700-phpapp02/85/Lsb-slides-1-320.jpg)
