black-box testing is a type of software testing in which the tester is not concerned with the internal knowledge or implementation details of the software but rather focuses on validating the functionality based on the provided specifications or requirements.

1. SHADES OF ANALYSIS
1. WHITE BOX TESTING
2. BLACK BOX TESTING
3. GREY BOX TESTING

2. WHITE BOX TESTING
• White box testing, also known as glass box testing, clear box
testing, or structural testing,
• It is a software testing technique where the internal structure,
design, and implementation of the software under test are
examined.
• In white box testing, testers have access to the source code
and are able to design test cases based on the internal logic of
the application.

3. • In the security world, this can also be thought of as an insider
attack.
• The tester has access to the source code and design
documentation. This allows the tester to be efficient.
• He can threat-model the system or do a line-by-line code review,
looking for information to guide the selection of test data.
• White box testing is the most efficient way to find security
vulnerabilities. More information allows quicker and more complete
generation of interfaces to test.
• It also gives you an accurate picture of the system’s security
because it doesn’t rely on security by obscurity, which is the hope
that attackers will never discover information about how a system
works.
• Security by obscurity is not real security. You should always assume
that eventually all information about a system will be discovered or

4. • A well-designed and well-implemented system will still be secure.
This is why good crypto algorithms can be published for review.
They don’t rely on privacy for security.
• “Risk-Based Security Testing: Prioritizing Security Testing with
Threat Modeling,” described a process for working with architects
and component owners to determine the software’s design.
• That process should be followed before any white box testing takes
place. Doing so will uncover the software’s attack surface and will
show you what functionality was put in place to mitigate the risks
that the attack surface poses. White box testing tests these
mitigations.

5. • A sample mitigation is the security mechanism put in place to
counter the threat of inadequate randomness in a session identifier.
• After this mitigation is discovered through the threat-modeling
process, the security tester can inspect the code that generates the
session identifier.
• This usually is not enough because small mistakes that can be
difficult to determine by inspecting the code can cause the
randomness to be weak and vulnerable.
• A white box test would be to automate the process of creating a new
session and to record the session identifiers that are generated.
These identifiers can then be subjected to a mathematical analysis
to see if they are truly random.

6. Key aspects of white box testing include:
• Understanding of Internal Logic
• Code Coverage
• Test Case Design
• Knowledge of Programming Languages
• Unit Testing
• Integration Testing

7. Benefits:
White box testing can uncover hidden errors, ensure
comprehensive test coverage, and help improve the quality and
reliability of the software.
Challenges:
White box testing requires detailed knowledge of the
internal implementation, which may not always be feasible or
practical. Additionally, changes to the code may necessitate
updates to the test cases, making maintenance challenging.

8. Black box testing
• Black box testing is a software testing technique where the
internal structure, design, and implementation of the software
under test are not known to the tester.
• Instead, the tester interacts with the software as an end-user
would, focusing on testing the functionality and behavior of the
system without knowledge of its internal workings.

9. • Black box testing involves examining the system as an
outsider would, using tools to detect the attack surface and
probe the system for internal information.
• With no internal knowledge of the system, the tester builds an
understanding of the system.
• Information leakage is especially important to the black box
tester because it helps him build more understanding than he
would otherwise get by manipulating a leak-free program.
• Many testers swear by black boxing techniques to complement white
box testing. If too much emphasis is given to specifications and design
documentation, the tester may miss parts of the system that were
built incorrectly or were not included in the documentation.

10. • This out-of-spec functionality may harbor security flaws that must be
discovered.
• Black box testing lets the tester probe all of the attack surface and
generate test data for functionality that may not be in the design.
• A common mistake is not removing debug-only commands before
production. Another is to throw in last-minute functionality that is
not properly documented in the design.
• Black box testing can find flaws in these cases that would otherwise
go unnoticed by the white box tester.
• Black box testing can be used when a system is deliberately using
security by obscurity to help protect information, as is often the case
in digital rights management (DRM) systems.

11. • Software-only DRM is impossible to completely secure because the
attacker has control of the system where the DRM software is
executing.
• The best a DRM manufacturer can hope for is to raise the bar for a
successful attack so high that a would-be attacker gives up.
• Black box testing by a skilled reverse-engineering team is often used
to test the strength of the obscurity used.
• This typically requires expertise outside the norm for a quality
assurance team and can be expensive. However, for DRM systems, it
is necessary to demonstrate the efficacy of the obfuscation used.

12. Key aspects of black box testing include:
• Independence from Internal Implementation
• Test Case Design
• Functional Testing- validating the system against the
functional specification and requirements
• Boundary Value Analysis-testing the values at extreme
boundaries of input domain
• User-Centric Perspective
• Regression Testing-process of testing the modified parts of
the code

13. Benefits:
Black box testing provides an unbiased assessment of
the software's functionality, regardless of its internal
implementation. It helps uncover defects and ensures that the
software meets the specified requirements and user
expectations.
Challenges:
Black box testing may not uncover certain types of
defects related to the internal logic or implementation of the
software. Additionally, designing comprehensive test cases
without knowledge of the internal structure can be challenging.

14. GREY BOX TESTING
• Grey box testing in software security involves assessing the
security of a system with partial knowledge of its internal
workings.
• This approach combines elements of both white box and black
box testing techniques to identify vulnerabilities and
weaknesses in the software from a security standpoint.
• Ideally both white box and black box techniques are used
during security testing.
• White box testing is used to discover flaws in functionality that
were specified in the design and development.
• Black box testing is used to discover flaws without having
access to these application internals. Sometimes this
combination is called gray box testing.

15. • The application security tester typically performs gray box testing to
find vulnerabilities in software. Flaws due to design and flaws due
to unspecified functionality are equally important to discover. Because
the source code is available to the security tester, it should be used to
improve productivity.
• Running the software under test in a debugger is the perfect way to
meld a running black box test with the source code to give the
tester the advantage of the gray box.
• In the Windows world, Microsoft Developer Studio is typically the
debugger of choice if debug symbols and source code are available.
• It allows the tester to navigate easily through the stack and memory
to explore complex variables such as classes and structures.
• In the UNIX world, gdb is typically used for this purpose.

16. • As soon as the software is running in the debugger, the normal tools of
black box testing can be brought to bear on the executing program.
Fuzzers or automated regression suites are typically deployed for this.
• The tester can set break points on lines of code that are dangerous to see
if they can be reached with external input to the program.
• These dangerous lines of code can be discovered with a code review or
simply by greping or searching the code.
• An example of a dangerous code is a sprintf statement in a C program that
has a %s in the format string.
• If the source buffer that is copied to the destination buffer is too large, a
buffer overflow condition occurs. But not every sprintf statement with this
condition can be exploited. Gray box testing lets you find the lines of code
that are truly exploitable.

17. EXAMPLE
?SomeFunction(char *input)
{
..
char dest[50];
..
sprintf(dest, "The
output is %s", input);
..
}

18. • The preceding might be an exploitable problem, but sometimes you
cannot easily determine from source code if that is true.
• The path user input takes to get to the input variable can be
complex. A break point can be set on this and any other similar lines
of code.
• Then the fuzzer or another automated test can be run to see if any
of the break points are hit.
• If a break point is hit, the stack can be inspected to see what the
control flow was to reach that point and if any validation is placed on
the input.
• If there isn’t, the test input can be manipulated to determine if the
vulnerable line of code can be reached with data that causes an
exploitable condition.

19. • If issues are discovered during the development of the code, it is
almost always a good idea to just fix the potentially exploitable
code.
• But many development teams are loathe to fix issues that might be
a problem in code that has already shipped to the customer or that
was put into production due to the patching cost.
• A gray box approach to finding vulnerabilities gives critical
exploitability information to the development team.
• This helps them make an informed decision about fixing a potential
vulnerability in the code.

20. Setting Up a Lab for Testing
• A dedicated lab with the appropriate hardware and software tools is
essential for efficient security testing. Many of these tools are open-
source, so you can use them as a starting point for building customer
clients.
• VMware or other virtualization programs should be used to have
standard installations of operating systems available.
• A file server can store images. This way, the OS can be brought to a
known good state.

21. • It is possible through fuzzing and other automated testing to corrupt
configuration files, the Windows Registry, or even system binaries.
• If this happens, you don’t want to suffer through a full reinstall.
• A quick refresh with a saved VMware image can return you to the
state you need to be in to continue testing.
Fuzzers
Open-Source
• SPIKE (fuzzer framework with some prebuild protocol fuzzers)
• Peachfuzz (another fuzzer framework)
• Mangle (HTML fuzzer)
• FileFuzz (file fuzzer)
Commercial
• beStorm
• Codenomicon

22. Sniffers
Open-Source
• tcpdump
• Ethereal (probably the only one you will need)
• Snort (an intrusion detection system [IDS] that can be used as a sniffer)
• Dsniff (a collection of tools that can sniff on a switched network)
Commercial
• AeroPeek
• EtherPeak
• Agilent Network Analyzer
• Network General Sniffer (the original packet sniffer)

23. Debuggers
Open-Source
• gdb
• Shareware
• OllyDbg
Commercial
• Microsoft Developer Studio
• SoftIce

24. Hardware
• Obviously you need hardware that meets the requirements for the
software to be tested. This may mean having Intel x86, PowerPC, or SPARC
systems available.
• Auditor (collection of security tools)
• BackTrack (a combination of WHAX and Auditor)
• Knoppix (large generic bootable CD)
Commercial Testing Appliances
• Mu-4000 Security Analyzer
• Spirent ThreatEx