SAST Review for Python

What is SAST?

Static Application Security Testing (SAST) is a way to analyze code without actually running it. Tools can analyze dependency graphs, control flow, complexity and measure various security aspects.

Static Code Analysis examines the code without executing it, and can be thought of as white-box testing. In contrast, dynamic code analysis analyzes while it is running and it's more like black-box testing.

Reviewing source code to identify vulnerability sources is the base workings of the service. SAST tools focus on the code contents of an application. It focuses on white-box testing where you know the entire environment and can recreate problems. In contrast, DAST runs while the application runs which is more like black-box testing.

Purpose of SAST

Early detection
SAST tools analyze source code to identify security flaws before the software is run. Integrating SAST in the dev lifecycle can detect vulnerabilities in the coding phase. This can include well-known problems with SQL injections, cross-site scripting and buffer overflows, or more niche ones. It's a proactive risk approach.

Cost Efficiency
Addressing security flaws during the development is obviously much less costly than dealing with them after release. Fixing security flaws after product release involves patching live systems, downtime, and resource allocation. It can easily become more than 10 times as expensive compared to finding the problem during development.

Compliance and Standards
SAST is a starting step to help organizations meet compliance requirements and adhere to industry standards. GDPR regulation on information privacy might the most well-known.

Being compliant helps with avoid legal penalties, pass security audits, and maintain trust from customers. Other than being secure, of course.

Code Quality
Similarly to linters, SAST tools contributes to enhancing the overall code quality due how a developer has to structure their code to meet security requirements. It can cover many of the things a linter checks as well.

Key Terminology

SAST

Static Application Security Testing
Code scanning of source code of a project to identify common sources of vulnerabilities.

CVE

Common Vulnerability and Exposure
A database of publicly disclosed cybersecurity vulnerabilities with currently over 240.000 records. Commonly used as ground truth for discussing vulnerabilities containing very specific vulnerabilities.

CWE

Common Weakness Enumeration
Categorizes vulnerabilities in more general terms, like "weak password" or "SQL injection". CVEs are mapped to these umbrella terms.

CVSS

Common Vulnerability Scoring System
Scoring system for CVEs from a scale from 0.0 to 10.0. Important factors are how easy the vulnerability is to reproduce and how severe it is.

DAST

Dynamic Application Security Testing
Scanning while code is running. Often discussed in relation to SAST.

SCA

Software Composition Analysis
Verification of third-party components.

Linter vs. SAST

Linters focuses on syntax, SAST on insecure code patterns.

	Linter	SAST
Main focus	Syntax, code style, adherence to code standards	Code patterns, common vulnerabilities, insecure code
Data flow analysis	No	Yes
Issues detected	Unused code, syntax errors, stylistic errors, suspicious constructs	Security vulnerabilities
Usage frequency	During development	During audits or CI/CD pipelines
Outcome	Improved readability and consistent code style and quality	Improves security by detecting vulnerabilities early

How SAST Works Under the Hood

Parsing and Abstract Syntax Trees
Parsing the source code to create an AST is a process that involves converting the code into a tree-like structure representing how the code is organized syntactically.

Each node corresponds to a syntactic construct in the code (variables, functions, operators, and control flow statements).

This way irrelevant details like formatting and comments are removed, and only the syntactic structure remains.

Pattern Matching
Using the AST it's easier to try to find code patterns that may be vulnerable. The resulting AST object can be looped through to find specific sequences or functions that are known to be vulnerable.

This relies on databases of known issues.

Data Flow
Data flow analysis examines how variables are assigned, modified and used in the program. This way you can build a model of the state and analyze how data propagates through the system.

Taint Analysis
Type of data flow analysis that tries to identify how tainted data can affect the security of the application. Data going through user inputs, file reads and network responses are marked as "tainted" and the analysis checks if they reach sensitive parts of the codebase such as database queries or system commands.

Custom Rules
Additional custom rules can be added, simple or complex.

Simple: internal function is deprecated, don't use it
Complex: rules for bugs that only happens to a module between 1 and 2 pm

Helpful to make all developers follow a company guideline.

Challenges with SAST Scanning

False positives and negatives
Scanning code is an optimization problem in the way that you have to have a balance between how thorough a tool analyzes and the accuracy. Tools need to be sensitive enough to detect real vulnerabilities but not overwhelm a developer with false positives.

A false positive occurs when a SAST tool reports a vulnerability that doesn't exist. This can lead to wasted resources, and over time cause developers to ignore warnings.

False negatives are real vulnerabilities that are ignored.

A mix of algorithms and heuristics are needed to have a good balance. Tools often provide options and custom rules for teams to adjust themselves.

Third-party dependencies
Python code often uses many different libraries, which is a great part of the open source community, but introduces challenges with scanning code. Scanning the source code of all imported libraries is not always possible or even beneficial. For these it is often best to rely on the CVEs and the SAST tool correctly flagging a CVE if an unsecure package is used.

Complex Code Structure
Large codebases with intricate logic and multiple layers of abstraction might pose a challenge to a simple static tool.

Tracing execution through deeply nested calls and recursive algorithms, and accounting for multiple options and race conditions in asynchronous programming are additional challenging fields for SAST tools.

Python specifics
Python handles a lot of stuff at runtime, and not in advance, which can lead to uncertainty in static tools.
Also, eval() and exec() are the classic functions that can easily become huge vulnerabilities.

Rating Vulnerabilities

Let's take a quick look at how vulnerabilities are ranked.

Severity levels
Different communities use different ways of tagging their vulnerabilities. Most well-known is probably the Low-Medium-High-Crucial or just classic 1-10.
Although there are plenty of discussions on how numbering doesn't really make sense.

CVSS (Common Vulnerability Scoring System)
Here the 0.0 to 10.0 range matters which reflects the severity of a vulnerability, and then mapped to a rating (Low-Medium-High-Critical).

I'm not getting into the details of how these vulnerabilities are categorized, but things taken into consideration are:

Base characteristics
Characteristics changing over time
Characteristics unique to a user group

EPSS (Exploit Prediction Scoring System)
EPSS is an interesting framework that is designed to estimate the probability of a vulnerability to be exploited in the wild within a certain time. This is done by training machine learning models on historical data and vulnerability characteristics.

Other sources

NVD (National Vulnerability Database) provides CVEs with CVSS scores.
Community databases such as Github and security researchers contribute a lot too.

Tool Testing Results

Testing was performed on a repository inspired by the common series of purposefully vulnerable applications named Damn Vulnerable.

The repository in question: Damn Vulnerable Python Web App.

Results

SAST Review

SAST Analysis Levels

SAST is implemented early, and is ongoing while developing. The occurence of false positives is often the main driver or killer for many developers. It overall tries to reduce the risk of downtime in applications or compromise of private information.

SAST quality is scored on a set of analysis levels:

Function - Instruction sequence
File or class - Program-code-template for object creation
Application - Multiple interacting programs

The scope of the analysis will affect how much the tool can help with contextual information, or use of consumed metadata.

Strengths:

Early fixes are cheap
Can cover 100% of source code
Also does quality and architectural testing
Scales well
Can run repeatedly

Weaknesses:

Output may be lacking in usability
Struggles with bugs related to authentication, access control and insecure cryptography
Early integration generates bugs in agile environments because developers focus first on features and delivery
Thousands of lines of code can generate many warnings and false positives, increasing investigation time
Struggles with code that can't be compiled

Tool Selection Criteria

Selection Criteria // Common Prioritization

Supports your programming language(s)
Ability to detect vulnerabilities based on known lists (OWASP, OSSTMM, CHECK, ...)
Accuracy (False positive/False negative rates)
Supports your framework
Availability as an IDE plugin
Ease of setup and use
Cost

Compliance Requirements

Regulatory, lawful and other enforced requirements
GDPR for data privacy.
HIPAA for health information privacy.
PCI DSS is an industry standard to all entities involved in payment card processing.
ISO/IEC 27001 is an international standard for information security management systems.
NIST guidelines provides a framework for best practices.

Failure to comply will result in fines, unsecure code and loss of trust.

Role of SAST
SAST ensures code meets the security standards, demonstrates a commitment to security, and is a risk management tool giving automated reports on security issues.