Reference

pynguin.cli

Pynguin is an automated unit test generation framework for Python.

This module provides the main entry location for the program execution from the command line.

pynguin.cli.main(argv=None)

Entry point for the CLI of the Pynguin automatic unit test generation framework.

This method behaves like a standard UNIX command-line application, i.e., the return value 0 signals a successful execution. Any other return value signals some errors. This is, e.g., the case if the framework was not able to generate one successfully running test case for the class under test.

Parameters:

argv (Optional[list[str]]) – List of command-line arguments

Return type:

int

Returns:

An integer representing the success of the program run. 0 means success, all non-zero exit codes indicate errors.

pynguin.configuration

Provides a configuration interface for the test generator.

class pynguin.configuration.Algorithm(value)

Different algorithms supported by Pynguin.

DYNAMOSA = 'DYNAMOSA'

The dynamic many-objective sorting algorithm (cf. Panichella et al. Automated test case generation as a many-objective optimisation problem with dynamic selection of the targets. TSE vol. 44 issue 2).

LLM = 'LLM'

This does not execute the usual test and assertion generation stages of Pynguin but queries an LLM and writes the resulting test cases to the output file.

Type:

Query a large language model for tests. NOTE

LLMOSA = 'LLMOSA'

The many-objective sorting algorithm with LLM.

MIO = 'MIO'

The MIO test suite generation algorithm (cf. Andrea Arcuri. Many Independent Objective (MIO) Algorithm for Test Suite Generation. Proc. SBSE 2017).

MOSA = 'MOSA'

The many-objective sorting algorithm (cf. Panichella et al. Reformulating Branch Coverage as a Many-Objective Optimization Problem. Proc. ICST 2015).

RANDOM = 'RANDOM'

A feedback-direct random test generation approach similar to the algorithm proposed by Randoop (cf. Pacheco et al. Feedback-directed random test generation. Proc. ICSE 2007).

RANDOM_TEST_CASE_SEARCH = 'RANDOM_TEST_CASE_SEARCH'

Performs random search on test cases.

RANDOM_TEST_SUITE_SEARCH = 'RANDOM_TEST_SUITE_SEARCH'

Performs random search on test suites.

WHOLE_SUITE = 'WHOLE_SUITE'

A whole-suite test generation approach similar to the one proposed by EvoSuite (cf. Fraser and Arcuri. EvoSuite: Automatic Test Suite Generation for Object-Oriented Software. Proc. ESEC/FSE 2011).

This algorithm can be modified to use an archive (cf. Rojas, José Miguel, et al. “A detailed investigation of the effectiveness of whole test suite generation.” Empirical Software Engineering 22.2 (2017): 852-893.), by using the following options: –use-archive True, –seed-from-archive True and –filter-covered-targets-from-test-cluster True.

class pynguin.configuration.AssertionGenerator(value)

Different approaches for assertion generation supported by Pynguin.

CHECKED_MINIMIZING = 'CHECKED_MINIMIZING'

All assertions that do not increase the checked coverage are removed.

LLM = 'LLM'

Use Large Language Model(LLM) for assertion generation.

MUTATION_ANALYSIS = 'MUTATION_ANALYSIS'

Use the mutation analysis approach for assertion generation.

NONE = 'NONE'

Do not create any assertions.

SIMPLE = 'SIMPLE'

Use the simple approach for primitive and none assertion generation.

class pynguin.configuration.Configuration(project_path, module_name, test_case_output, algorithm=Algorithm.DYNAMOSA, statistics_output=<factory>, stopping=<factory>, large_language_model=<factory>, string_statement=<factory>, seeding=<factory>, type_inference=<factory>, pynguinml=<factory>, test_creation=<factory>, generator_selection=<factory>, search_algorithm=<factory>, mio=<factory>, random=<factory>, to_cover=<factory>, ignore_modules=<factory>, ignore_methods=<factory>, subprocess=False, subprocess_if_recommended=True, local_search=<factory>, use_master_worker=True, filesystem_isolation=False)

General configuration for the test generator.

algorithm: Algorithm = 'DYNAMOSA'

The algorithm that shall be used for generation.

filesystem_isolation: bool = False

Whether to use filesystem isolation during test execution. Filesystem isolation provides some safety by tracking and controlling file operations, e.g. preventing file deletion of non-created files, but adds performance overhead.

ignore_methods: list[str]

Ignore the methods specified here from the module analysis.

ignore_modules: list[str]

Ignore the modules specified here from the module analysis.

large_language_model: LLMConfiguration

Large Language Model(LLM) configuration.

local_search: LocalSearchConfiguration

Local search configuration.

mio: MIOConfiguration

Configuration used for the MIO algorithm.

module_name: str

Name of the module for which the generator shall create tests.

project_path: str

Path to the project the generator shall create tests for.

pynguinml: PynguinMLConfiguration

PynguinML configuration.

random: RandomConfiguration

Configuration used for the RANDOM algorithm.

search_algorithm: SearchAlgorithmConfiguration

Search algorithm configuration.

seeding: SeedingConfiguration

Seeding configuration.

statistics_output: StatisticsOutputConfiguration

Statistic Output configuration.

stopping: StoppingConfiguration

Stopping configuration.

string_statement: StringStatementConfiguration

String statement configuration.

subprocess: bool = False

Run the test generation in a subprocess.

subprocess_if_recommended: bool = True

Use the subprocess mode if the SUT analysis recommends it based on used C extension modules.

test_case_output: TestCaseOutputConfiguration

Configuration for how test cases should be output.

test_creation: TestCreationConfiguration

Test creation configuration.

to_cover: ToCoverConfiguration

Configuration of which code elements are included or excluded as coverage goals.

type_inference: TypeInferenceConfiguration

Type inference configuration.

use_master_worker: bool = True

Use master-worker architecture for fault tolerance.

class pynguin.configuration.CoverageMetric(value)

The different available coverage metrics available for optimisation.

BRANCH = 'BRANCH'

Calculate how many of the possible branches in the code were executed

CHECKED = 'CHECKED'

Calculate how many of the possible lines in the code are checked by an assertion.

LINE = 'LINE'

Calculate how many of the possible lines in the code were executed

class pynguin.configuration.ExportStrategy(value)

Contains all available export strategies.

These strategies allow to export the generated test cases in different styles, such as the style of the PyTest framework. Setting the value to NONE will prevent exporting of the generated test cases (only reasonable for benchmarking, though).

NONE = 'NONE'

Do not export test cases at all.

PY_TEST = 'PY_TEST'

Export tests in the style of the PyTest framework.

class pynguin.configuration.GeneratorSelectionConfiguration(generator_selection_algorithm=Selection.RANK_SELECTION, generator_selection_bias=1.7, generator_any_distance=30, generator_not_constructor_penalty=10.0, generator_param_penalty=1.0, generator_hierarchy_penalty=1.0, generator_any_type_penalty=100.0)

Configuration related to selecting a generator for a certain type.

generator_any_distance: int = 30

The distance to and from the any type to any other type in the type hierarchy. As the any type is a super- and subtype of all type but it is most likely not the type we want, we want to punish it in the type selection process.

generator_any_type_penalty: float = 100.0

Penalty for selecting a generator that returns only Any. While Any is always a valid type, we still want to avoid it as a generator.

generator_hierarchy_penalty: float = 1.0

Penalty for the distance between the generators type and the desired type. The penalty is multiplied with the distance in the type hierarchy.

generator_not_constructor_penalty: float = 10.0

Penalty for selecting a generator that is not a __init__ method.

generator_param_penalty: float = 1.0

Penalty for each parameter of the generator.

generator_selection_algorithm: Selection = 'RANK_SELECTION'

The selection algorithm to use for selecting the generator for a statement. Currently only RANK_SELECTION is supported.

generator_selection_bias: float = 1.7

The bias to use for rank selection when selecting the generator for a statement. Expects values in [1.0, 2.0]. 1.0 is random selection, higher values bias towards higher ranked generators. A value above 2.0 leads to cutting off the lower ranked generators.

class pynguin.configuration.LLMConfiguration(api_key='', model_name='gpt-4o-mini', llm_url='', temperature=0.8, hybrid_initial_population=False, llm_test_case_percentage=0.5, enable_response_caching=False, call_llm_for_uncovered_targets=False, coverage_threshold=1, call_llm_on_stall_detection=False, max_plateau_len=25, max_llm_interventions=1)

Configuration for the LLM.

api_key: str = ''

The api key to call OpenAI LLM with.

call_llm_for_uncovered_targets: bool = False

Whether to call the LLM for the uncovered targets initially.

call_llm_on_stall_detection: bool = False

Whether to call the LLM for the uncovered targets in coverage stalls.

coverage_threshold: float = 1

The coverage threshold when to call the LLM for low-coverage targets. The value must be from [0.0, 1.0].

enable_response_caching: bool = False

Whether to enable caching for responses of the model.

hybrid_initial_population: bool = False

Whether to include the LLM test cases in the initial population.

llm_test_case_percentage: float = 0.5

The percentage of LLM test cases in on the initial population. The value must be from [0.0, 1.0].

llm_url: str = ''

Base URL for the LLM API endpoint (leave empty for default OpenAI).

max_llm_interventions: int = 1

The maximum number of allowed LLM interventions.

max_plateau_len: int = 25

The number of iterations to allow before soliciting the LLM.

model_name: str = 'gpt-4o-mini'

The OpenAI Model to use for completions.

temperature: float = 0.8

The temperature to use when querying the model. The value must be from [0.0, 1.0].

class pynguin.configuration.LocalSearchConfiguration(local_search=True, local_search_same_datatype=True, local_search_different_datatype=False, local_search_llm=False, local_search_primitives=True, local_search_collections=False, local_search_complex_objects=False, local_search_probability=0.02, local_search_time=5000, ls_int_delta_increasing_factor=2, ls_string_random_mutation_count=10, ls_random_parametrized_statement_call_count=10, ls_max_different_type_mutations=10, ls_different_type_primitive_probability=0.3, ls_different_type_collection_probability=0.3, ls_dict_max_insertions=10, ls_llm_whole_module=False)

Configurations for local search.

local_search: bool = True

Defines if local search is generally enabled or not

local_search_collections: bool = False

Toggles if local search for collection types is enabled or not

local_search_complex_objects: bool = False

Toggles if local search for complex object types is enabled or not

local_search_different_datatype: bool = False

Defines if local search with different datatypes is enabled or not

local_search_llm: bool = False

Defines if local search with llm is enabled or not

local_search_primitives: bool = True

Toggles if local search for primitive types is enabled or not

local_search_probability: float = 0.02

Probability of starting local search on the specific candidate

local_search_same_datatype: bool = True

Defines if local search within same datatype is enabled or not

local_search_time: int = 5000

The budget which each local search iteration has in ms.

ls_dict_max_insertions: int = 10

Max number of unsuccessful insertions of entries.

ls_different_type_collection_probability: float = 0.3

Weight of generating collection statements when changing the statement type.

ls_different_type_primitive_probability: float = 0.3

Weight of generating primitive statements when changing the statement type.

ls_int_delta_increasing_factor: int = 2

The factor which defines how much the delta should increase for each iteration for integer local search.

ls_llm_whole_module: bool = False

Whether to provide the whole module or selected functions/methods to the LLM for context when performing llm local search.

ls_max_different_type_mutations: int = 10

The max number of searches for different datatypes

ls_random_parametrized_statement_call_count: int = 10

The number of mutations made to a parametrized statement call to determine if it’s worth to further do local search

ls_string_random_mutation_count: int = 10

The number of mutations made to the string to determine if it’s worth to completely mutate the string.

class pynguin.configuration.MIOConfiguration(initial_config=<factory>, focused_config=<factory>, exploitation_starts_at_percent=0.5)

Configuration that is specific to the MIO approach.

exploitation_starts_at_percent: float = 0.5

Percentage ]0,1] of search budget after which exploitation is activated, i.e., switching to focused phase.

focused_config: MIOPhaseConfiguration

Configuration to use in focused phase

initial_config: MIOPhaseConfiguration

Configuration to use before focused phase.

class pynguin.configuration.MIOPhaseConfiguration(number_of_tests_per_target, random_test_or_from_archive_probability, number_of_mutations)

Configuration for a phase of MIO.

number_of_mutations: int

Number of mutations allowed to be done on the same individual before sampling a new one.

number_of_tests_per_target: int

Number of test cases for each target goal to keep in an archive.

random_test_or_from_archive_probability: float

Probability [0,1] of sampling a new test at random or choose an existing one in an archive.

class pynguin.configuration.Minimization(test_case_minimization_strategy=MinimizationStrategy.CASE, test_case_minimization_direction=MinimizationDirection.BACKWARD)

Configuration for test case minimization.

test_case_minimization_direction: MinimizationDirection = 'BACKWARD'

Direction to apply for minimizing test cases.

test_case_minimization_strategy: MinimizationStrategy = 'CASE'

Strategy to apply for minimizing test cases to remove redundant statements while preserving fitness.

class pynguin.configuration.MinimizationDirection(value)

Directions for test case minimization.

Either start with the first statement and try removing it and all forward dependent statements, or start with the last statement and try removing it and all backward dependent statements.

BACKWARD = 'BACKWARD'

Apply backward minimization, which removes statements from the end to the beginning.

FORWARD = 'FORWARD'

Apply forward minimization, which removes statements from the beginning to the end.

class pynguin.configuration.MinimizationStrategy(value)

Different strategies for minimizing test cases.

CASE = 'CASE'

Apply minimization at the test case level.

COMBINED = 'COMBINED'

Apply minimization at the test case level but check against all test cases in the suite.

NONE = 'NONE'

Do not apply any minimization.

SUITE = 'SUITE'

Apply minimization at the test suite level, i.e., remove entire test cases that do not contribute to the coverage of the module under test.

class pynguin.configuration.MutationStrategy(value)

Different strategies for creating mutants.

Only respected when using the MUTATION_ANALYSIS approach for assertion generation.

BETWEEN_OPERATORS = 'BETWEEN_OPERATORS'

Higher order mutation strategy BetweenOperators. (cf. Mateo et al. Validating Second-Order Mutation at System Level. Article. IEEE Transactions on SE 39.4 2013)

EACH_CHOICE = 'EACH_CHOICE'

Higher order mutation strategy EachChoice. (cf. Mateo et al. Validating Second-Order Mutation at System Level. Article. IEEE Transactions on SE 39.4 2013)

FIRST_ORDER_MUTANTS = 'FIRST_ORDER_MUTANTS'

Generate first order mutants.

FIRST_TO_LAST = 'FIRST_TO_LAST'

Higher order mutation strategy FirstToLast. (cf. Mateo et al. Validating Second-Order Mutation at System Level. Article. IEEE Transactions on SE 39.4 2013)

RANDOM = 'RANDOM'

Higher order mutation strategy Random. (cf. Mateo et al. Validating Second-Order Mutation at System Level. Article. IEEE Transactions on SE 39.4 2013)

class pynguin.configuration.PynguinMLConfiguration(ml_testing_enabled=False, constraints_path='', dtype_mapping_path='', constructor_function='', constructor_function_parameter='', max_ndim=4, max_shape_dim=4, ignore_constraints_probability=0.25)

Configurations for PynguinML, specifically for machine learning API testing.

constraints_path: str = ''

Directory path to YAML/JSON constraints files.

constructor_function: str = ''

Optional constructor function to build tensors (e.g., torch.tensor or tensorflow.convert_to_tensor). Must accept a np.ndarray as input. Note: Also set ‘constructor_function_parameter’ to specify which parameter receives the np.ndarray.

constructor_function_parameter: str = ''

Name of the parameter in the constructor function that receives the np.ndarray. Must be set when ‘constructor_function’ is specified.

dtype_mapping_path: str = ''

Path to a YAML/JSON file that maps library-specific datatypes to NumPy datatypes. Used by PynguinML for input generation. Specify only valid NumPy dtypes without the “np.” prefix. Example: “torch.int32” -> “int32”.

ignore_constraints_probability: float = 0.25

Probability of ignoring constraints and falling back to Pynguin’s default behavior when constructing statements.

max_ndim: int = 4

Maximum number of dimensions of the tensors. If set too big, it can cause memory errors due to large tensor sizes.

max_shape_dim: int = 4

Maximum size of a dimension of a shape. If set too big, it can cause memory errors due to large tensor sizes.

ml_testing_enabled: bool = False

Enables ML-based test generation using API constraints. When set to True, PynguinML will use constraints and other ML-specific behaviours during input generation.

Note: ‘constraints_path’ must also be set to locate the constraints files.

class pynguin.configuration.RandomConfiguration(max_sequence_length=10, max_sequences_combined=10)

Configuration that is specific to the RANDOM approach.

max_sequence_length: int = 10

The maximum length of sequences that are generated, 0 means infinite.

max_sequences_combined: int = 10

The maximum number of combined sequences, 0 means infinite.

class pynguin.configuration.SearchAlgorithmConfiguration(min_initial_tests=1, max_initial_tests=10, population=10, chromosome_length=48, chop_max_length=True, elite=1, crossover_rate=0.648, test_insertion_probability=0.1, test_delete_probability=0.3333333333333333, test_change_probability=0.3333333333333333, test_insert_probability=0.3333333333333333, statement_insertion_probability=0.5, random_perturbation=0.2, change_parameter_probability=0.1, tournament_size=4, rank_bias=1.68, selection=Selection.RANK_SELECTION, use_archive=False, filter_covered_targets_from_test_cluster=False, number_of_mutations=3)

General configuration for search algorithms.

change_parameter_probability: float = 0.1

Probability of replacing parameters when mutating a method or constructor statement in a test case. Expects values in [0,1]

chop_max_length: bool = True

Chop statements after exception if length has reached maximum

chromosome_length: int = 48

Maximum length of chromosomes during search

crossover_rate: float = 0.648

Probability of crossover

elite: int = 1

Elite size for search algorithm

filter_covered_targets_from_test_cluster: bool = False

Focus search by filtering out elements from the test cluster when they are fully covered.

max_initial_tests: int = 10

Maximum number of tests in initial test suites

min_initial_tests: int = 1

Minimum number of tests in initial test suites

number_of_mutations: int = 3

Number of mutations that should be applied in one breeding step.

population: int = 10

Population size of genetic algorithm

random_perturbation: float = 0.2

Probability to replace a primitive with a random new value rather than adding a delta.

rank_bias: float = 1.68

Bias for better individuals in rank selection

selection: Selection = 'RANK_SELECTION'

The selection operator for genetic algorithms.

statement_insertion_probability: float = 0.5

Initial probability of inserting a new statement in a test case

test_change_probability: float = 0.3333333333333333

Probability of changing statements during mutation

test_delete_probability: float = 0.3333333333333333

Probability of deleting statements during mutation

test_insert_probability: float = 0.3333333333333333

Probability of inserting new statements during mutation

test_insertion_probability: float = 0.1

Initial probability of inserting a new test in a test suite

tournament_size: int = 4

Number of individuals for tournament selection.

use_archive: bool = False

Some algorithms can be enhanced with an optional archive, e.g. Whole Suite -> Whole Suite + Archive. Use this option to enable the usage of an archive. Algorithms that always use an archive are not affected by this option.

class pynguin.configuration.SeedingConfiguration(seed=1780383204402602139, constant_seeding=True, initial_population_seeding=False, initial_population_data='', seeded_testcases_reuse_probability=0.9, initial_population_mutations=0, dynamic_constant_seeding=True, seeded_primitives_reuse_probability=0.2, seeded_dynamic_values_reuse_probability=0.6, seed_from_archive=False, seed_from_archive_probability=0.2, seed_from_archive_mutations=3, max_dynamic_length=1000, max_dynamic_pool_size=50)

Configuration related to seeding.

constant_seeding: bool = True

Should the generator use a static constant seeding technique to improve constant generation?

dynamic_constant_seeding: bool = True

Enables seeding of constants at runtime.

initial_population_data: str = ''

The path to the file with the pre-existing tests. The path has to include the file itself.

initial_population_mutations: int = 0

Number of how often the testcases collected by initial population seeding should be mutated to promote diversity

initial_population_seeding: bool = False

Should the generator use previously existing testcases to seed the initial population?

max_dynamic_length: int = 1000

Maximum length of strings/bytes that should be stored in the dynamic constant pool.

max_dynamic_pool_size: int = 50

Maximum number of constants of the same type that should be stored in the dynamic constant pool.

seed: int = 1780383204402602139

A predefined seed value for the random number generator that is used.

seed_from_archive: bool = False

When sampling new test cases reuse some from the archive, if one is used.

seed_from_archive_mutations: int = 3

Number of mutations applied when sampling from the archive.

seed_from_archive_probability: float = 0.2

Instead of creating a new test case, reuse a covering solution from the archive, iff an archive is used.

seeded_dynamic_values_reuse_probability: float = 0.6

Probability of using dynamically seeded values when a primitive seeded value will be used.

seeded_primitives_reuse_probability: float = 0.2

Probability for using seeded primitive values instead of randomly generated ones.

seeded_testcases_reuse_probability: float = 0.9

Probability of using seeded testcases when initial population seeding is enabled.

class pynguin.configuration.Selection(value)

Different selection algorithms to select from.

RANDOM_SELECTION = 'RANDOM_SELECTION'

Random selection.

RANK_SELECTION = 'RANK_SELECTION'

Rank selection.

TOURNAMENT_SELECTION = 'TOURNAMENT_SELECTION'

Tournament selection. Use tournament_size to set size.

class pynguin.configuration.StatisticsBackend(value)

The different available statistics backends to write statistics.

CONSOLE = 'CONSOLE'

Write statistics to the standard out.

CSV = 'CSV'

Write statistics to a CSV file.

NONE = 'NONE'

Do not write any statistics.

class pynguin.configuration.StatisticsOutputConfiguration(report_dir='pynguin-report', statistics_backend=StatisticsBackend.CSV, timeline_interval=1000000000, timeline_interpolation=True, coverage_metrics=<factory>, output_variables=<factory>, configuration_id='', run_id='', project_name='', create_coverage_report=False, type_guess_top_n=10, store_test_before_execution=False)

Configuration related to output.

configuration_id: str = ''

Label that identifies the used configuration of Pynguin. This is only done when running experiments.

coverage_metrics: list[CoverageMetric]

List of coverage metrics that are optimised during the search

create_coverage_report: bool = False

Create a coverage report for the tested module. This can be helpful to find hard to cover parts because Pynguin measures coverage on bytecode level which might yield different results when compared with other tools, e.g., Coverage.py.

output_variables: list[RuntimeVariable]

List of variables to output to the statistics backend.

project_name: str = ''

Label that identifies the project name of Pynguin. This is useful when running experiments.

report_dir: str = 'pynguin-report'

Directory in which to put HTML and CSV reports

run_id: str = ''

Id of the cluster run. Useful for finding the log entries that belong to a certain result.

statistics_backend: StatisticsBackend = 'CSV'

Which backend to use to collect data

store_test_before_execution: bool = False

When enabled, each test case is written to a file before execution and removed after successful completion. If a test causes a crash, the file remains. Use with caution: Converting pynguin internal representation to a pytest test case and writing it to a file is expensive and leads to lower iterations/coverage.

timeline_interpolation: bool = True

Interpolate timeline values

timeline_interval: int = 1000000000

Time interval in nano-seconds for timeline statistics, i.e., we select a data point after each interval. This can be interpolated, if there is no exact value stored at the time-step of the interval, see timeline_interpolation. The default value is every 1.00s.

type_guess_top_n: int = 10

When exporting type guesses for parameters, how many guesses per parameter should be exported? Expects positive integers.

class pynguin.configuration.StoppingConfiguration(maximum_search_time=-1, maximum_test_executions=-1, maximum_statement_executions=-1, maximum_slicing_time=600, maximum_iterations=-1, maximum_test_execution_timeout=5, maximum_coverage=100, maximum_coverage_plateau=-1, minimum_coverage=100, minimum_plateau_iterations=-1, maximum_memory=3000, test_execution_time_per_statement=1)

Configuration related to when Pynguin should stop.

Note that these are mostly soft-limits rather than hard limits, because the search algorithms only check the condition at the start of each algorithm iteration.

maximum_coverage: int = 100

The maximum percentage of coverage after which the generation shall stop.

maximum_coverage_plateau: int = -1

Maximum number of algorithm iterations without coverage change before the algorithms stops.

maximum_iterations: int = -1

Maximum iterations

maximum_memory: int = 3000

Maximum memory usage in MB after which the generation shall stop. Expects values in MB greater than 0 of -1 to disable the check.

maximum_search_time: int = -1

Time (in seconds) that can be used for generating tests.

maximum_slicing_time: int = 600

Time budget (in seconds) that can be used for slicing.

maximum_statement_executions: int = -1

Maximum number of test cases to be executed.

maximum_test_execution_timeout: int = 5

The maximum time (in seconds) after which a test case times out.

maximum_test_executions: int = -1

Maximum number of test cases to be executed.

minimum_coverage: int = 100

Minimum coverage for the plateau-based stopping condition. Expects values larger than 0 but less than 100 to activate the stopping condition; also requires the setting of minimum_plateau_iterations.

minimum_plateau_iterations: int = -1

Minimum iterations without a coverage change to stop early. Expects values larger than 0; also requires the setting of minimum_coverage.

test_execution_time_per_statement: int = 1

The time (in seconds) per statement that a test is allowed to run (up to maximum_test_execution_timeout).

class pynguin.configuration.StringStatementConfiguration(random_string_weight=0.3, faker_string_weight=0.3, fandango_string_weight=0.4, fandango_faker_string_weight=0.0)

Configuration related to string statements.

faker_string_weight: float = 0.3

Probability use the Faker generator for string statement generation.

fandango_faker_string_weight: float = 0.0

Probability use the Fandango + Faker generator for string statement generation.

fandango_string_weight: float = 0.4

Probability use the Fandango generator for string statement generation.

random_string_weight: float = 0.3

Probability use the random generator for string statement generation.

class pynguin.configuration.SubtypeInferenceStrategy(value)

The different available type-inference strategies.

NONE = 'NONE'

Do not infer subtypes.

STRING = 'STRING'

Infer subtypes for strings.

class pynguin.configuration.TestCaseOutputConfiguration(output_path, crash_path='', export_strategy=ExportStrategy.PY_TEST, max_length_test_case=2500, assertion_generation=AssertionGenerator.MUTATION_ANALYSIS, allow_stale_assertions=False, mutation_strategy=MutationStrategy.FIRST_ORDER_MUTANTS, mutation_order=1, post_process=True, minimization=<factory>, float_precision=0.01, format_with_black=True, no_xfail=False)

Configuration related to test case output.

allow_stale_assertions: bool = False

Allow assertion on things that did not change between statement executions.

assertion_generation: AssertionGenerator = 'MUTATION_ANALYSIS'

The generator that shall be used for assertion generation.

crash_path: str = ''

Path to an output folder for the generated test cases that caused a crash. Only works when running in a subprocess. If set to “”, the are stored in the output_path.

export_strategy: ExportStrategy = 'PY_TEST'

The export strategy determines for which test-runner system the generated tests should fit.

float_precision: float = 0.01

Precision to use in float comparisons and assertions

format_with_black: bool = True

Format the generated test cases using black.

max_length_test_case: int = 2500

The maximum number of statement in as test case (normal + assertion statements)

minimization: Minimization

Strategy to apply for minimizing test cases.

mutation_order: int = 1

The order of the generated higher order mutants in the mutation analysis assertion generation method.

mutation_strategy: MutationStrategy = 'FIRST_ORDER_MUTANTS'

The strategy that shall be used for creating mutants in the mutation analysis assertion generation method.

no_xfail: bool = False

If True, unexpected exceptions will be wrapped with pytest.raises() instead of marking the test with @pytest.mark.xfail(strict=True). Use this for better mutation scores.

output_path: str

Path to an output folder for the generated test cases.

post_process: bool = True

Should the results be post processed? For example, truncate test cases after statements that raise an exception.

class pynguin.configuration.TestCreationConfiguration(max_recursion=10, max_delta=20, max_int=2048, string_length=20, bytes_length=20, collection_size=5, primitive_reuse_probability=0.5, object_reuse_probability=0.9, none_weight=1, any_weight=5, original_type_weight=5, type_tracing_weight=10, type_tracing_kept_guesses=2, wrap_var_param_type_probability=0.7, negate_type=0.1, skip_optional_parameter_probability=0.7, max_attempts=1000, insertion_uut=0.5, max_size=100, use_random_object_for_call=0.1)

Configuration related to test creation.

any_weight: float = 5

Weight to use Any as parameter type during test generation. Expects values > 0.

bytes_length: int = 20

Maximum length of randomly generated bytes

collection_size: int = 5

Maximum length of randomly generated collections

insertion_uut: float = 0.5

Score for selection of insertion of UUT calls

max_attempts: int = 1000

Number of attempts when generating an object before giving up

max_delta: int = 20

Maximum size of delta for numbers during mutation

max_int: int = 2048

Maximum size of randomly generated integers (minimum range = -1 * max)

max_recursion: int = 10

Recursion depth when trying to create objects in a test case.

max_size: int = 100

Maximum number of test cases in a test suite

negate_type: float = 0.1

When inferring a type from proxies, it may also be desirable to negate the chosen type, e.g., such that an instance check or a getattr() evaluate to False. Expects values in [0,1]

none_weight: float = 1

Weight to use None as parameter type during test generation. Expects values > 0.

object_reuse_probability: float = 0.9

Probability to reuse an existing object in a test case, if available. Expects values in [0,1]

original_type_weight: float = 5

Weight to use the originally annotated type as parameter type during test generation. Expects values > 0.

primitive_reuse_probability: float = 0.5

Probability to reuse an existing primitive in a test case, if available. Expects values in [0,1]

skip_optional_parameter_probability: float = 0.7

Probability to skip an optional parameter, i.e., do not fill such a parameter.

string_length: int = 20

Maximum length of randomly generated strings

type_tracing_kept_guesses: int = 2

Amount of kept recently guessed types per parameter, when type tracing is used.

type_tracing_weight: float = 10

Weight to use the type guessed from type tracing as parameter type during test generation. Expects values > 0.

use_random_object_for_call: float = 0.1

When adding or modifying a call on an object, use a random modifier instead of only modifiers for that type. Expects values in [0, 1].

wrap_var_param_type_probability: float = 0.7

Probability to wrap the type required for a *arg or **kwargs parameter in a list or dict, respectively. Expects values in [0,1]

class pynguin.configuration.ToCoverConfiguration(only_cover=<factory>, no_cover=<factory>, enable_inline_pynguin_no_cover=True, enable_inline_pragma_no_cover=True)

Configuration of which code elements are included or excluded as coverage goals.

Pynguin instruments only the respective parts of the SUT module for coverage measurement. All configuration options are resolved as SUT line numbers which must not overlap (‘only’ and ‘no’ at the same time).

enable_inline_pragma_no_cover: bool = True

Enable inline pragma: no cover.

enable_inline_pynguin_no_cover: bool = True

Enable inline pynguin: no cover.

no_cover: list[str]

The list of space-separated qualified function, method or class names to not cover. Automatically include the methods of the ignore_methods argument.

only_cover: list[str]

The list of space-separated qualified function, method or class names to only cover.

class pynguin.configuration.TypeInferenceConfiguration(type_inference_strategy=TypeInferenceStrategy.TYPE_HINTS, type_tracing=0.0, subtype_inference=SubtypeInferenceStrategy.NONE, type_tracing_subtype_weight=0.3, type_tracing_argument_type_weight=0.5, type_tracing_attribute_weight=0.2, typeevalpy_json_path='')

Configuration related to type inference.

subtype_inference: SubtypeInferenceStrategy = 'NONE'

The strategy for subtype-inference that shall be used.

type_inference_strategy: TypeInferenceStrategy = 'TYPE_HINTS'

The strategy for type-inference that shall be used

type_tracing: bool | float = 0.0

Probability to trace usage of parameters with unknown types to improve type guesses during test execution. Type tracing requires a separate second. The value should be a float in [0,1]. Boolean is kept for backwards compatibility as Python internally converts True to 1.0 and False to 0.0 anyways.

type_tracing_argument_type_weight: float = 0.5

Weight for selecting the argument type inference strategy for type selection during type tracing.

type_tracing_attribute_weight: float = 0.2

Weight for selecting the attribute table inference strategy for type selection during type tracing.

type_tracing_subtype_weight: float = 0.3

Weight for selecting the subtype inference strategy for type selection during type tracing.

typeevalpy_json_path: str = ''

Path to a TypeEvalPy JSON file containing type information to use during test generation. If provided, the file will be parsed and the type information will be used to improve type inference.

class pynguin.configuration.TypeInferenceStrategy(value)

The different available type-inference strategies.

LLM = 'LLM'

Use an LLM to infer types for the module under test.

LLM_WITH_SUBTYPES = 'LLM_WITH_SUBTYPES'

Use an LLM to infer types and subtypes for the module under test.

NONE = 'NONE'

Ignore any type information given in the module under test.

TYPEEVALPY = 'TYPEEVALPY'

Use TypeEvalPy data only for type inference.

TYPE_HINTS = 'TYPE_HINTS'

Use type information from type hints in the module under test.

pynguin.generator

Pynguin is an automated unit test generation framework for Python.

The framework generates unit tests for a given Python module. For this it supports various approaches, such as a random approach, similar to Randoop or a whole-suite approach, based on a genetic algorithm, as implemented in EvoSuite. The framework allows to export test suites in various styles, i.e., using the unittest library from the Python standard library or tests in the style used by the PyTest framework.

Pynguin is supposed to be used as a standalone command-line application but it can also be used as a library by instantiating this class directly.

class pynguin.generator.ReturnCode(value)

Return codes for Pynguin to signal result.

FINAL_METRICS_TRACKING_FAILED = 3

Symbolises that the final metrics tracking failed.

NO_TESTS_GENERATED = 2

Symbolises that no test could be generated.

OK = 0

Symbolises that the execution ended as expected.

SETUP_FAILED = 1

Symbolises that the execution failed in the setup phase.

pynguin.generator.run_pynguin()

Run the test generation.

The result of the test generation is indicated by the resulting ReturnCode.

Return type:

ReturnCode

Returns:

See ReturnCode.

Raises:

ConfigurationException – In case the configuration is illegal

pynguin.generator.set_configuration(configuration)

Initialises the test generator with the given configuration.

Parameters:

configuration (Configuration) – The configuration to use.

Return type:

None

pynguin.analyses.module

Provides analyses for the subject module, based on the module and its AST.

class pynguin.analyses.module.CallableData(accessible, tree, description, cyclomatic_complexity)

Provides all information on callables.

While the accessible is available for every callable, the other fields are only filled for methods that are available in (Python) source code because their information is retrieved from the abstract syntax tree.

accessible

the accessible object itself

tree

the AST of the callable, if any

description

the function description of the callable, if any

cyclomatic_complexity

the McCabe cyclomatic complexity of the callable, if any

class pynguin.analyses.module.FilteredModuleTestCluster(delegate, archive, subject_properties, targets)

A test cluster wrapping another test cluster.

Delegates most methods to the wrapped delegate. This cluster filters out accessible objects under test that are already fully covered, in order to focus the search on areas that are not yet fully covered.

property accessible_objects_under_test: OrderedSet[GenericAccessibleObject]

Provides all accessible objects under test.

add_accessible_object_under_test(objc, data)

Add accessible object to the objects under test.

Parameters:

• objc (GenericAccessibleObject) – The accessible object

• data (CallableData) – The function-description data

Return type:

None

add_generator(generator)

Add the given accessible as a generator.

Parameters:

generator (GenericAccessibleObject) – The accessible object

Return type:

None

add_ml_data(obj, data)

Provides ML data for a accessible.

Return type:

None

add_modifier(typ, obj)

Add a modifier.

A modifier is something that can be used to modify the given type, for example, a method.

Parameters:

• typ (TypeInfo) – The type that can be modified

• obj (GenericAccessibleObject) – The accessible that can modify

Return type:

None

property function_data_for_accessibles: dict[GenericAccessibleObject, CallableData]

Provides all function data for all accessibles.

property generators: dict[ProperType, OrderedSet[GenericAccessibleObject]]

Provides all available generators.

get_all_generatable_types()

Provides all types that can be generated.

This includes primitives and collections.

Returns:

DAR202

Return type:

A list of all types that can be generated # noqa

get_generators_for(typ)

Retrieve all known generators for the given type.

Parameters:

typ (ProperType) – The type we want to have the generators for

Return type:

OrderedSet[GenericAccessibleObject]

Returns:

The set of all generators for that type.

get_ml_data_for(obj)

Provides ML data for a accessible.

Return type:

MLCallableData | None

get_modifiers_for(typ)

Get all known modifiers for a type.

Parameters:

typ (ProperType) – The type

Returns:

DAR202

Return type:

The set of all accessibles that can modify the type # noqa

get_random_accessible()

Provides a random accessible of the unit under test.

Returns:

DAR202

Return type:

A random accessible, or None if there is none # noqa

get_random_call_for(typ)

Get a random modifier for the given type.

Parameters:

typ (ProperType) – The type

Returns:

DAR202

Return type:

A random modifier for that type # noqa

Raises:

ConstructionFailedException – if no modifiers for the type exist# noqa: DAR402

property linenos: int

Provide the number of source code lines.

log_cluster_statistics()

Log the signatures of all seen callables.

Return type:

None

property modifiers: dict[TypeInfo, OrderedSet[GenericAccessibleObject]]

Provides all available modifiers.

num_accessible_objects_under_test()

Provide the number of accessible objects under test.

Useful to check whether there is even something to test.

Return type:

int

on_target_covered(target)

A callback function to get informed by an archive when a target is covered.

Parameters:

target (TestCaseFitnessFunction) – The newly covered target

Return type:

None

select_concrete_type(typ)

Select a concrete type from the given type.

This is required, for example, when handling union types. Currently, only unary types, Any, and Union are handled.

Parameters:

typ (ProperType) – An optional type

Returns:

DAR202

Return type:

An optional type # noqa

track_statistics_values(tracking_fun)

Track statistics values from the test cluster and its items.

Parameters:

tracking_fun (Callable[[RuntimeVariable, Any], None]) – The tracking function as a callback.

Return type:

None

property type_system: TypeSystem

Provides the inheritance graph.

update_parameter_knowledge(accessible, param_name, knowledge)

Update the knowledge about the parameter of the given accessible.

Parameters:

• accessible (GenericCallableAccessibleObject) – the accessible that was observed.

• param_name (str) – the parameter name for which we have new information.

• knowledge (UsageTraceNode) – the new information.

Return type:

None

update_return_type(accessible, new_type)

Update the return for the given accessible to the new seen type.

Parameters:

• accessible (GenericCallableAccessibleObject) – the accessible that was observed

• new_type (ProperType) – the new return type

Return type:

None

class pynguin.analyses.module.MLCallableData(parameters, generation_order)

Provides ML-specific information on callables.

parameters

A dictionary of parameters, if any

generation_order

The generation order of the callable (can be empty)

class pynguin.analyses.module.ModuleTestCluster(linenos)

A test cluster for a module.

Contains all methods/constructors/functions and all required transitive dependencies.

property accessible_objects_under_test: OrderedSet[GenericAccessibleObject]

Provides all accessible objects under test.

add_accessible_object_under_test(objc, data)

Add accessible object to the objects under test.

Parameters:

• objc (GenericAccessibleObject) – The accessible object

• data (CallableData) – The function-description data

Return type:

None

add_generator(generator)

Add the given accessible as a generator.

Parameters:

generator (GenericAccessibleObject) – The accessible object

Return type:

None

add_ml_data(obj, data)

Provides ML data for a accessible.

Return type:

None

add_modifier(typ, obj)

Add a modifier.

A modifier is something that can be used to modify the given type, for example, a method.

Parameters:

• typ (TypeInfo) – The type that can be modified

• obj (GenericAccessibleObject) – The accessible that can modify

Return type:

None

property function_data_for_accessibles: dict[GenericAccessibleObject, CallableData]

Provides all function data for all accessibles.

property generators: dict[ProperType, OrderedSet[GenericAccessibleObject]]

Provides all available generators.

get_all_generatable_types()

Provides all types that can be generated.

This includes primitives and collections.

Returns:

DAR202

Return type:

A list of all types that can be generated # noqa

get_generators_for(typ)

Retrieve all known generators for the given type.

Parameters:

typ (ProperType) – The type we want to have the generators for

Return type:

OrderedSet[GenericAccessibleObject]

Returns:

The set of all generators for that type.

get_ml_data_for(obj)

Provides ML data for a accessible.

Return type:

MLCallableData | None

get_modifiers_for(typ)

Get all known modifiers for a type.

Parameters:

typ (ProperType) – The type

Returns:

DAR202

Return type:

The set of all accessibles that can modify the type # noqa

get_random_accessible()

Provides a random accessible of the unit under test.

Returns:

DAR202

Return type:

A random accessible, or None if there is none # noqa

get_random_call_for(typ)

Get a random modifier for the given type.

Parameters:

typ (ProperType) – The type

Returns:

DAR202

Return type:

A random modifier for that type # noqa

Raises:

ConstructionFailedException – if no modifiers for the type exist# noqa: DAR402

property linenos: int

Provide the number of source code lines.

log_cluster_statistics()

Log the signatures of all seen callables.

Return type:

None

property modifiers: dict[TypeInfo, OrderedSet[GenericAccessibleObject]]

Provides all available modifiers.

num_accessible_objects_under_test()

Provide the number of accessible objects under test.

Useful to check whether there is even something to test.

Return type:

int

select_concrete_type(typ)

Select a concrete type from the given type.

This is required, for example, when handling union types. Currently, only unary types, Any, and Union are handled.

Parameters:

typ (ProperType) – An optional type

Returns:

DAR202

Return type:

An optional type # noqa

track_statistics_values(tracking_fun)

Track statistics values from the test cluster and its items.

Parameters:

tracking_fun (Callable[[RuntimeVariable, Any], None]) – The tracking function as a callback.

Return type:

None

property type_system: TypeSystem

Provides the type system.

Returns:

The type system.

update_parameter_knowledge(accessible, param_name, knowledge)

Update the knowledge about the parameter of the given accessible.

Parameters:

• accessible (GenericCallableAccessibleObject) – the accessible that was observed.

• param_name (str) – the parameter name for which we have new information.

• knowledge (UsageTraceNode) – the new information.

Return type:

None

update_return_type(accessible, new_type)

Update the return for the given accessible to the new seen type.

Parameters:

• accessible (GenericCallableAccessibleObject) – the accessible that was observed

• new_type (ProperType) – the new return type

Return type:

None

class pynguin.analyses.module.SignatureInfo(annotated_parameter_types=<factory>, guessed_parameter_types=<factory>, partial_type_matches=<factory>, annotated_return_type=None, recorded_return_type=None)

Another utility class to group information per callable.

class pynguin.analyses.module.TestCluster

Interface for a test cluster.

abstract property accessible_objects_under_test: OrderedSet[GenericAccessibleObject]

Provides all accessible objects under test.

abstract add_accessible_object_under_test(objc, data)

Add accessible object to the objects under test.

Parameters:

• objc (GenericAccessibleObject) – The accessible object

• data (CallableData) – The function-description data

Return type:

None

abstract add_generator(generator)

Add the given accessible as a generator.

Parameters:

generator (GenericAccessibleObject) – The accessible object

Return type:

None

abstract add_ml_data(obj, data)

Provides ML data for a accessible.

Return type:

None

abstract add_modifier(typ, obj)

Add a modifier.

A modifier is something that can be used to modify the given type, for example, a method.

Parameters:

• typ (TypeInfo) – The type that can be modified

• obj (GenericAccessibleObject) – The accessible that can modify

Return type:

None

abstract property function_data_for_accessibles: dict[GenericAccessibleObject, CallableData]

Provides all function data for all accessibles.

abstract property generators: dict[ProperType, OrderedSet[GenericAccessibleObject]]

Provides all available generators.

abstract get_all_generatable_types()

Provides all types that can be generated.

This includes primitives and collections.

Returns:

DAR202

Return type:

A list of all types that can be generated # noqa

abstract get_generators_for(typ)

Retrieve all known generators for the given type.

Parameters:

typ (ProperType) – The type we want to have the generators for

Return type:

OrderedSet[GenericAccessibleObject]

Returns:

The set of all generators for that type.

abstract get_ml_data_for(generic_accessible)

Provides ML data for a accessible.

Return type:

MLCallableData | None

abstract get_modifiers_for(typ)

Get all known modifiers for a type.

Parameters:

typ (ProperType) – The type

Returns:

DAR202

Return type:

The set of all accessibles that can modify the type # noqa

abstract get_random_accessible()

Provides a random accessible of the unit under test.

Returns:

DAR202

Return type:

A random accessible, or None if there is none # noqa

abstract get_random_call_for(typ)

Get a random modifier for the given type.

Parameters:

typ (ProperType) – The type

Returns:

DAR202

Return type:

A random modifier for that type # noqa

Raises:

ConstructionFailedException – if no modifiers for the type exist# noqa: DAR402

abstract property linenos: int

Provide the number of source code lines.

abstract log_cluster_statistics()

Log the signatures of all seen callables.

Return type:

None

abstract property modifiers: dict[TypeInfo, OrderedSet[GenericAccessibleObject]]

Provides all available modifiers.

abstract num_accessible_objects_under_test()

Provide the number of accessible objects under test.

Useful to check whether there is even something to test.

Return type:

int

abstract select_concrete_type(typ)

Select a concrete type from the given type.

This is required, for example, when handling union types. Currently, only unary types, Any, and Union are handled.

Parameters:

typ (ProperType) – An optional type

Returns:

DAR202

Return type:

An optional type # noqa

abstract track_statistics_values(tracking_fun)

Track statistics values from the test cluster and its items.

Parameters:

tracking_fun (Callable[[RuntimeVariable, Any], None]) – The tracking function as a callback.

Return type:

None

abstract property type_system: TypeSystem

Provides the inheritance graph.

abstract update_parameter_knowledge(accessible, param_name, knowledge)

Update the knowledge about the parameter of the given accessible.

Parameters:

• accessible (GenericCallableAccessibleObject) – the accessible that was observed.

• param_name (str) – the parameter name for which we have new information.

• knowledge (UsageTraceNode) – the new information.

Return type:

None

abstract update_return_type(accessible, new_type)

Update the return for the given accessible to the new seen type.

Parameters:

• accessible (GenericCallableAccessibleObject) – the accessible that was observed

• new_type (ProperType) – the new return type

Return type:

None

class pynguin.analyses.module.TypeGuessingStats(number_of_constructors=0, signature_infos=<factory>)

Class to gather some type guessing related statistics.

pynguin.analyses.module.analyse_module(parsed_module, type_inference_strategy=TypeInferenceStrategy.TYPE_HINTS)

Analyses a module to build a test cluster.

Parameters:

• parsed_module (_ModuleParseResult) – The parsed module

• type_inference_strategy (TypeInferenceStrategy) – The type inference strategy to use.

Return type:

ModuleTestCluster

Returns:

A test cluster for the module

pynguin.analyses.module.collect_provider_metrics(typ_provider)

Collects metrics from the given type inference provider.

currently, this only works for LLM-based providers.

When using an LLM-based provider, collect the raw inferred parameter strings per callable and the annotated parameter strings as JSON and store in the LLMInferredSignatures runtime variable.

Other providers default all metrics to zero.

pynguin.analyses.module.generate_test_cluster(module_name, type_inference_strategy=TypeInferenceStrategy.TYPE_HINTS)

Generates a new test cluster from the given module.

Parameters:

• module_name (str) – The name of the root module

• type_inference_strategy (TypeInferenceStrategy) – Which type-inference strategy to use

Return type:

ModuleTestCluster

Returns:

A new test cluster for the given module

pynguin.analyses.module.get_type_provider(type_inference_strategy, module, type_system)

Get the initialised inference provider for the given strategy.

Parameters:

• type_inference_strategy (TypeInferenceStrategy) – The type inference strategy to use

• module (ModuleType) – The module to analyse (only needed for LLM-based inference)

• type_system (TypeSystem) – The type system to use

Return type:

InferenceProvider

Returns:

The type inference provider for the given strategy

pynguin.analyses.module.import_module(module_name)

Imports a module by name.

Unlike the built-in importlib.import_module(), this function also supports importing module aliases.

Parameters:

module_name (str) – The fully-qualified name of the module

Return type:

ModuleType

Returns:

The imported module

pynguin.analyses.module.parse_module(module_name)

Parses a module and extracts its module-type and AST.

If the source code is not available it is not possible to build an AST. In this case the respective field of the _ModuleParseResult will contain the value None. This is the case, for example, for modules written in native code, for example, in C.

Parameters:

module_name (str) – The fully-qualified name of the module

Return type:

_ModuleParseResult

Returns:

A tuple of the imported module type and its optional AST

pynguin.analyses.module.read_module_ast(module_path, module_name)

Reads the AST of the module and returns it along with its source code.

Parameters:

• module_path (str) – The path of the module.

• module_name (str) – The name of the module.

Raises:

• OSError – if the module file cannot be read.

• AstroidError – if an error occurs during the creation of the AST.

Return type:

tuple[Module, str]

Returns:

A tuple containing the AST and the source code.

pynguin.analyses.typesystem

Provides analyses for a module’s type information.

class pynguin.analyses.typesystem.AnyType

The Any Type.

accept(visitor)

Accept a type visitor.

Parameters:

visitor (TypeVisitor[TypeVar(T)]) – the visitor

Return type:

TypeVar(T)

class pynguin.analyses.typesystem.CSVString

A string that represents comma-separated values.

Examples: “value1,value2,value3”, “itemA,itemB,itemC”.

class pynguin.analyses.typesystem.EmailString

A string that follows the general pattern of an email address.

Example: “name@domain.edu”.

class pynguin.analyses.typesystem.HexadecimalString

A string that represents a hexadecimal number.

Examples: “0x1A3F”, “1a3f”, “0XABCDEF”.

class pynguin.analyses.typesystem.IPv4String

A string that represents an IPv4 address.

Examples: “192.168.0.0”, “74.5.123.31”.

class pynguin.analyses.typesystem.IPv6String

A string that represents an IPv6 address.

Examples: “2001:0db8:85a3:0000:0000:8a2e:0370:7334”, “fe80::1ff:fe23:4567:890a”.

class pynguin.analyses.typesystem.ISOColorString

A string that represents a hexadecimal color code.

Examples: “#FFFFFF”, “#000000”, “#FF5733”.

class pynguin.analyses.typesystem.ISODateString

A string that represents a date in ISO 8601 format.

Examples: “2023-10-05”, “1999-12-31”.

class pynguin.analyses.typesystem.ISOTimeString

A string that represents a time in ISO 8601 format.

Examples: “14:30:00”, “23:59:59”.

class pynguin.analyses.typesystem.InferredSignature(signature, original_return_type, original_parameters, type_system, parameters_for_statistics=<factory>, return_type_for_statistics=AnyType())

Encapsulates the types inferred for a method.

get_parameter_types(signature_memo)

Get a possible type signature for the parameters.

This method may choose a random type signature, or return the original one or create one based on the observed knowledge.

Parameters:

signature_memo (dict[InferredSignature, dict[str, ProperType]]) – A memo that stores already chosen signatures, so that we don’t choose another signature in the same run. This is required for certain operations in the test factory to be consistent.

Return type:

dict[str, ProperType]

Returns:

A dict of chosen parameter types for each parameter.

log_stats_and_guess_signature(is_constructor, callable_full_name, stats)

Logs some statistics and creates a guessed signature.

Parameters annotated with Any could not be guessed.

Parameters:

• callable_full_name (str) – The full, unique name of the callable.

• is_constructor (bool) – does this signature to a constructor?

• stats (TypeGuessingStats) – stats object to log to.

Return type:

None

class pynguin.analyses.typesystem.Instance(typ, args=None)

An instance type of form C[T1, …, Tn].

C is a class. Args can be empty.

accept(visitor)

Accept a type visitor.

Parameters:

visitor (TypeVisitor[TypeVar(T)]) – the visitor

Return type:

TypeVar(T)

class pynguin.analyses.typesystem.NamedDefaultDict

A default dictionary that automatically creates nodes for keys.

Default dict which automatically creates a UsageTraceNode for each requested and non-existing key.

class pynguin.analyses.typesystem.NoneType

The None type.

accept(visitor)

Accept a type visitor.

Parameters:

visitor (TypeVisitor[TypeVar(T)]) – the visitor

Return type:

TypeVar(T)

class pynguin.analyses.typesystem.NumericString

A string that only contains numerical characters and an optional decimal point.

Examples: “123”, “123.45”, “0.001”

class pynguin.analyses.typesystem.PhoneNumberString

A string that represents a phone number.

Examples: “+1-800-555-1234”, “(123) 456-7890”.

class pynguin.analyses.typesystem.ProperType

Base class for all types. Might have to add another layer, like mypy’s Type?.

All subclasses of this class are immutable.

abstract accept(visitor)

Accept a type visitor.

Parameters:

visitor (TypeVisitor[TypeVar(T)]) – the visitor

Return type:

TypeVar(T)

class pynguin.analyses.typesystem.SHA256String

A string that represents a SHA-256 hash.

Examples: “e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855”.

class pynguin.analyses.typesystem.StringSubtype(regex)

A subtype of str determined by its regex.

TODO(lk): Add support to all visitors.

accept(visitor)

Accept a type visitor.

Parameters:

visitor (TypeVisitor[TypeVar(T)]) – the visitor

Return type:

TypeVar(T)

class pynguin.analyses.typesystem.TupleType(args, *, unknown_size=False)

Tuple type Tuple[T1, …, Tn].

Note that tuple is a special case and intentionally not Instance(TypeInfo(tuple)) because tuple is varargs generic.

accept(visitor)

Accept a type visitor.

Parameters:

visitor (TypeVisitor[TypeVar(T)]) – the visitor

Return type:

TypeVar(T)

class pynguin.analyses.typesystem.TypeInfo(raw_type)

A small wrapper around type, i.e., classes.

Corresponds 1:1 to a class.

static get_dunder_value_from_type(typ, name)

Get the dunder value with the given name from the given type.

If the given type has no dunder attribute with the given name, we fall back to using the type of the given typ (== a value in this case). This worked for the UnionType.

Parameters:

• typ (type) – The type from which to get the attribute.

• name (str) – The name of the dunder attribute to get.

Return type:

str

Returns:

The value of the dunder attribute.

static to_full_name(typ)

Get the full name of the given type.

While type has a __name__, __qualname__ and __module__ attribute, UnionType does not. This caused a crash which is resolved by special handling of UnionType.

Parameters:

typ – The type for which we want a full name.

Return type:

str

Returns:

The fully qualified name

class pynguin.analyses.typesystem.TypeReprVisitor

A simple visitor to create a repr from a proper type.

visit_any_type(left)

Visit the Any type.

Parameters:

left (AnyType) – the Any type

Return type:

str

Returns:

result of the visit

visit_instance(left)

Visit an instance.

Parameters:

left (Instance) – instance

Return type:

str

Returns:

result of the visit

visit_none_type(left)

Visit the None type.

Parameters:

left (NoneType) – the None type

Return type:

str

Returns:

result of the visit

visit_tuple_type(left)

Visit a tuple type.

Parameters:

left (TupleType) – tuple

Return type:

str

Returns:

result of the visit

visit_union_type(left)

Visit a union.

Parameters:

left (UnionType) – union

Return type:

str

Returns:

result of the visit

visit_unsupported_type(left)

Visit unsupported type.

Parameters:

left (Unsupported) – unsupported

Return type:

str

Returns:

result of the visit

class pynguin.analyses.typesystem.TypeStringVisitor

A simple visitor to convert a proper type to a string.

visit_any_type(left)

Visit the Any type.

Parameters:

left (AnyType) – the Any type

Return type:

str

Returns:

result of the visit

visit_instance(left)

Visit an instance.

Parameters:

left (Instance) – instance

Return type:

str

Returns:

result of the visit

visit_none_type(left)

Visit the None type.

Parameters:

left (NoneType) – the None type

Return type:

str

Returns:

result of the visit

visit_tuple_type(left)

Visit a tuple type.

Parameters:

left (TupleType) – tuple

Return type:

str

Returns:

result of the visit

visit_union_type(left)

Visit a union.

Parameters:

left (UnionType) – union

Return type:

str

Returns:

result of the visit

visit_unsupported_type(left)

Visit unsupported type.

Parameters:

left (Unsupported) – unsupported

Return type:

str

Returns:

result of the visit

class pynguin.analyses.typesystem.TypeSystem

Implements Pynguin’s internal type system.

Provides a simple inheritance graph relating various classes using their subclass relationships. Note that parents point to their children.

This is also the central system to store/handle type information.

add_subclass_edge(*, super_class, sub_class)

Add a subclass edge between two types.

Parameters:

• super_class (TypeInfo) – superclass

• sub_class (TypeInfo) – subclass

Return type:

None

convert_type_hint(hint, unsupported=AnyType())

Converts a type hint to a proper type.

Python’s builtin functionality makes handling types during runtime really hard, because 1) this is not intended to be used at runtime and 2) there are a lot of different notations, due to the constantly evolving type hint system. We also cannot easily use mypy’s type abstraction because it is 1) strongly encapsulated and not part of mypy’s public API and 2) is designed to be used for static type checking. This method tries to translate type hints into our own type abstraction in order to make handling types less painful.

This conversion is naive when compared to what sophisticated type checkers like mypy do, but it is hopefully sufficient for our purposes. This method only handles a very small subset of the types that we may encounter in the wild, but at least it allows use to better reason about types. This should be extended in the future to handle more cases.

Parameters:

• hint (Any) – The type hint

• unsupported (ProperType) – The type to use when encountering an unsupported type construct

Return type:

ProperType

Returns:

A proper type.

property dot: str

Create dot representation of this graph.

Returns:

A dot string.

enable_numeric_tower()

Enable the numeric tower on this type system.

find_by_attribute(attr)

Search for all types that have the given attribute.

Parameters:

attr (str) – the attribute to search for.

Return type:

OrderedSet[TypeInfo]

Returns:

All types (or supertypes thereof) who have the given attribute.

find_type_info(full_name)

Find typeinfo for the given name.

Parameters:

full_name (str) – The name to search for.

Return type:

TypeInfo | None

Returns:

Type info, if any.

get_all_types()

Provides a list of all known types.

Return type:

list[TypeInfo]

Returns:

A list of all known types.

get_shortest_path_length(start, end)

Get the shortest path length between two types.

I tried adding @functools.lru_cache to this function does not improve performance, because we cache subtype_distance anyway. I tried using a _shortest_path_length_cache[(start, end)] along with using shortest_path_length(graph, source), which returns all nodes reachable from start and its length. However, probably the overhead from caching and having many different starting nodes make this slower than just using shortest_path_length with a target node (which can also leverage the quicker bidirectional_shortest_path internally). I also tried to come up with an own variant of nx.single_shortest_path_length that would stop at the target node, thinking the overhead maybe comes form deeper traversal, but this did not achieve better performance either. Thus, I suggest we stick to the simple version.

Parameters:

• start (TypeInfo) – The start type

• end (TypeInfo) – The end type

Return type:

int | None

Returns:

The shortest path length between the two types or None if no path exists.

get_subclasses(klass)

Provides all descendants of the given type. Includes klass.

Parameters:

klass (TypeInfo) – The class whose subtypes we want to query.

Return type:

OrderedSet[TypeInfo]

Returns:

All subclasses including klass

get_superclasses(klass)

Provides all ancestors of the given class.

Parameters:

klass (TypeInfo) – The class whose supertypes we want to query.

Return type:

OrderedSet[TypeInfo]

Returns:

All superclasses including klass

get_type_outside_of(klasses)

Find a type that does not belong to the given types or any subclasses.

Parameters:

klasses (OrderedSet[TypeInfo]) – The classes to exclude

Return type:

OrderedSet[TypeInfo]

Returns:

A set of klasses that don’t belong the given ones.

infer_signature(method, type_hint_provider)

Infers the method signature using the given type hint provider.

Parameters:

• method (Callable) – The callable

• type_hint_provider (Callable[[Callable], dict]) – A method that provides type hints for the given method.

Return type:

InferredSignature

Returns:

The inference result

infer_type_info(method, type_inference_provider)

Infers the type information for a callable.

Parameters:

• method (Callable) – The callable we try to infer type information for

• type_inference_provider (InferenceProvider) – The provider for type inference

Return type:

InferredSignature

Returns:

The inference result

is_maybe_subtype(left, right)

Is ‘left’ maybe a subtype of ‘right’?

This is a more lenient check than is_subtype. Consider a function that returns tuple[str | int | bytes, str | int | bytes]. Strictly speaking, we cannot use such a value as an argument for a function that requires an argument of type tuple[int, int]. However, it may be possible that the returned value is tuple[int, int], in which case it does work. This check only differs from is_subtype in how it handles Unions. Instead of requiring every type to be a subtype, it is sufficient that one type of the Union is a subtype.

Parameters:

• left (ProperType) – The left type

• right (ProperType) – The right type

Return type:

bool

Returns:

True, if left may be a subtype of right.

is_subclass(left, right)

Is ‘left’ a subclass of ‘right’?

Parameters:

• left (TypeInfo) – left type info

• right (TypeInfo) – right type info

Return type:

bool

Returns:

True, if there is a subclassing path from left to right.

is_subtype(left, right)

Is ‘left’ a subtype of ‘right’?

This check is more than incomplete, but it takes into account that anything is a subtype of AnyType.

See https://peps.python.org/pep-0483/ and https://peps.python.org/pep-0484/ for more details

Parameters:

• left (ProperType) – The left type

• right (ProperType) – The right type

Return type:

bool

Returns:

True, if left is a subtype of right.

make_instance(typ)

Create an instance from the given type.

Parameters:

typ (TypeInfo) – The type info.

Return type:

Instance | TupleType | NoneType

Returns:

An instance or TupleType

push_attributes_down()

Pushes attributes down in hierarchy.

We don’t want to see attributes multiple times, e.g., in subclasses, so only the first class in the hierarchy which adds the attribute should have it listed as an attribute, i.e., when searching for a class with that attribute we only want to retrieve the top-most class(es) in the hierarchy which define it, and not every (sub)class that inherited it.

Return type:

None

subtype_distance(supertype, subtype)

Computes the number of subclassing steps from the supertype to the subtype.

The number of subclassing steps is the shortest path length in the inheritance graph from the supertype to the subtype. The length from a type to itself is 0. If there is no path from the supertype to the subtype, i.e. if the supertype is not a supertype of the subtype, the method returns None. Similar to is_maybe_subtype, for unions it is sufficient that one type of the Union is a subtype and further the shortest path between all arguments of the Union is computed. See _SubtypeDistanceVisitor for special cases.

Examples

Assume (Super > Sub > SubSub). Super, Sub -> 1 Sub, Super -> None Super, Super -> 0 Super, SubSub -> 2

Parameters:

• supertype (ProperType) – The more general type.

• subtype (ProperType) – The more specific type.

Return type:

int | None

Returns:

The number of subclassing steps from left to right, or None if no path exists.

to_type_info(typ)

Find or create TypeInfo for the given type.

Parameters:

typ – The raw type we want to convert.

Return type:

TypeInfo

Returns:

A TypeInfo object.

try_to_load_type(candidate, globs)

Try to load the given type.

Parameters:

• candidate (str) – The type to load

• globs – The globals that should be used for loading.

Return type:

ProperType

Returns:

The loaded type or Any.

static type_hints_provider(method)

Provides PEP484-style type information, if available.

Parameters:

method (Callable) – The method for which we want type hints.

Return type:

dict[str, Any]

Returns:

A dict mapping parameter names to type hints.

wrap_var_param_type(typ, param_kind)

Wrap parameter types.

Wrap the parameter type of *args and **kwargs in List[…] or Dict[str, …], respectively.

Parameters:

• typ (ProperType) – The type to be wrapped.

• param_kind – the kind of parameter.

Return type:

ProperType

Returns:

The wrapped type, or the original type, if no wrapping is required.

class pynguin.analyses.typesystem.TypeVisitor

A type visitor.

Note that the parameter of the visit_* methods is called ‘left’, because it makes the implementations of _SubTypeVisitor and _MaybeSubTypeVisitor more clear and Python does not like changing the names of parameters in subclasses, thus we renamed them in this class.

abstract visit_any_type(left)

Visit the Any type.

Parameters:

left (AnyType) – the Any type

Return type:

TypeVar(T)

Returns:

result of the visit

abstract visit_instance(left)

Visit an instance.

Parameters:

left (Instance) – instance

Return type:

TypeVar(T)

Returns:

result of the visit

abstract visit_none_type(left)

Visit the None type.

Parameters:

left (NoneType) – the None type

Return type:

TypeVar(T)

Returns:

result of the visit

visit_string_subtype(left)

Visit a string subtype.

Parameters:

left (StringSubtype) – string subtype

Return type:

TypeVar(T)

Returns:

Result of the visit

abstract visit_tuple_type(left)

Visit a tuple type.

Parameters:

left (TupleType) – tuple

Return type:

TypeVar(T)

Returns:

result of the visit

abstract visit_union_type(left)

Visit a union.

Parameters:

left (UnionType) – union

Return type:

TypeVar(T)

Returns:

result of the visit

abstract visit_unsupported_type(left)

Visit unsupported type.

Parameters:

left (Unsupported) – unsupported

Return type:

TypeVar(T)

Returns:

result of the visit

class pynguin.analyses.typesystem.URLString

A string that represents a URL.

Examples: “http://example.com”, “https://www.domain.org/path?query=param”.

class pynguin.analyses.typesystem.UUIDString

A string that represents a UUID.

Examples: “123e4567-e89b-12d3-a456-426614174000”.

class pynguin.analyses.typesystem.UnionType(items)

The union type Union[T1, …, Tn] (at least one type argument).

accept(visitor)

Accept a type visitor.

Parameters:

visitor (TypeVisitor[TypeVar(T)]) – the visitor

Return type:

TypeVar(T)

class pynguin.analyses.typesystem.Unsupported

Marks an unsupported type in the type system.

Artificial type which represents a type that is currently not supported by our type abstraction. This is purely used for statistic purposes and should not be encountered during regular use.

accept(visitor)

Accept a type visitor.

Parameters:

visitor (TypeVisitor[TypeVar(T)]) – the visitor

Return type:

TypeVar(T)

pynguin.testcase.testcase

Provides an implementation for a test case.

class pynguin.testcase.testcase.TestCase(test_cluster)

An abstract base implementation for a test case.

Serves as an interface for test-case implementations

abstract accept(visitor)

Handles a test visitor.

Parameters:

visitor (TestCaseVisitor) – The test visitor to accept

Return type:

None

abstract add_statement(statement, position=-1)

Adds a new statement to the test case.

The optional position parameter specifies the position. If it is not given, the statement will be added to the end of the test case.

Parameters:

• statement (Statement) – The new statement

• position (int) – The optional position where to put the statement

Return type:

VariableReference | None

Returns: # noqa: DAR202

The return value of the statement. Notice that the test might choose to modify the statement you inserted. You should use the returned variable reference and not use references. Can be None, if this statement does not create a variable.

abstract add_statements(statements)

Adds a list of statements to the end of the test case.

Parameters:

statements (list[Statement]) – The list of statements to add

Return type:

None

abstract add_variable_creating_statement(statement, position=-1)

Overloaded version of add_statement that adds a statement.

Parameters:

• statement (VariableCreatingStatement) – The new statement

• position (int) – The optional position where to put the statement

Return type:

VariableReference

Returns: # noqa: DAR202

The return value of the statement. Notice that the test might choose to modify the statement you inserted. You should use the returned variable reference and not use references.

abstract append_test_case(test_case)

Appends a test case to this test case.

Parameters:

test_case (TestCase) – The test case to append

Return type:

None

abstract chop(pos)

Remove all statements after a given position.

Parameters:

pos (int) – The length of the test case after chopping

Return type:

None

abstract clone(limit=None)

Provides a deep copy of the test case.

Parameters:

limit (Optional[int]) – Clone this test case only up to the given number of statements.

Returns:

DAR202

Return type:

A deep copy of this test case # noqa

abstract contains(statement)

Determines whether or not the test case contains a specific statement.

Parameters:

statement (Statement) – The statement to search in the test case

Returns:

DAR202

Return type:

Whether or not the test case contains the statement # noqa

get_all_objects(position)

Get all objects that are defined up to the given position (exclusive).

Parameters:

position (int) – the position

Return type:

list[VariableReference]

Returns:

A list of all objects defined up to the given position

abstract get_assertions()

Get all assertions that exist for this test case.

Return type:

list[Assertion]

abstract get_dependencies(var)

Provides all variables on which var depends.

Parameters:

var (VariableReference) – the variable whose dependencies we are looking for.

Returns:

DAR202

Return type:

a set of variables on which var depends on. # noqa

abstract get_forward_dependencies(var)

Provides all variables that depend on var.

Parameters:

var (VariableReference) – the variable for which we look for all dependent statements of

Returns:

DAR202

Return type:

a set of variables that depend on var. # noqa

get_objects(parameter_type, position)

Provides a list of variable references satisfying a certain type.

If the position value is larger than the number of statements, only these statements will be considered. Otherwise, the first position statements of the test case will be considered.

If the type for which we search is not specified, all objects up to the given position are returned.

Parameters:

• parameter_type (ProperType) – The type of the parameter we search references for

• position (int) – The position in the statement list until we search

Return type:

list[VariableReference]

Returns:

A list of variable references satisfying the parameter type

get_random_object(parameter_type, position)

Get a random object of the given type up to the given position (exclusive).

Parameters:

• parameter_type (ProperType) – the parameter type

• position (int) – the position

Return type:

VariableReference

Returns:

A random object of given type up to the given position

Raises:

ConstructionFailedException – if no object could be found

abstract get_statement(position)

Provides access to a statement at a given position.

Parameters:

position (int) – The position of the statement in the test case

Returns:

DAR202

Return type:

The statement at the position # noqa

abstract has_statement(position)

Check if there is a statement at the given position.

Parameters:

position (int) – The index of the statement

Returns:

DAR202

Return type:

Whether or not there is a statement at the given position # noqa

static positions_to_remove(statement, dependencies)

Get the positions to remove and its forward dependencies in reverse order.

This is done to avoid index issues when removing multiple statements.

Parameters:

• statement (Statement) – The statement to remove

• dependencies (list[VariableReference]) – The forward dependencies of the statement

Return type:

list[int]

Returns:

A list of positions to remove

abstract remove(position)

Removes a statement a the given position.

Parameters:

position (int) – The position of the test case to be removed

Return type:

None

abstract remove_statement(statement)

Remove the given statement from this test case.

Parameters:

statement (Statement) – The statement to remove.

Return type:

None

abstract remove_statement_with_forward_dependencies(statement)

Removes the given statement along with all its forward dependencies.

Parameters:

statement (Statement) – The statement to remove

Return type:

list[int]

Returns:

A list of positions of statements that have been deleted

Raises:

ValueError – If the statement is not contained in the test case

abstract remove_with_forward_dependencies(position)

Removes a statement at the given position along with all its forward dependencies.

Parameters:

position (int) – The position of the statement to remove

Return type:

list[int]

Returns:

A list of positions of statements that have been deleted

Raises:

ValueError – If the position is out of bounds for this test case

abstract set_statement(statement, position)

Set new statement at position.

Parameters:

• statement (Statement) – the new statement

• position (int) – the position for the new statement

Returns:

DAR202

Return type:

A variable reference to the statements return value, if any # noqa

abstract size()

Provides the number of statements in the test case.

Returns:

DAR202

Return type:

The number of statements in the test case # noqa

abstract size_with_assertions()

Provides the number of statements and assertions in the test case.

Returns:

DAR202

Return type:

The number of statements and assertions in the test case # noqa

property statements: list[Statement]

Provides the list of statements in this test case.

Returns:

The list of statements in this test case

pynguin.testcase.defaulttestcase

Provides a default implementation of a test case.

class pynguin.testcase.defaulttestcase.DefaultTestCase(test_cluster)

A default implementation of a test case.

accept(visitor)

Handles a test visitor.

Parameters:

visitor (TestCaseVisitor) – The test visitor to accept

Return type:

None

add_statement(statement, position=-1)

Adds a new statement to the test case.

The optional position parameter specifies the position. If it is not given, the statement will be added to the end of the test case.

Parameters:

• statement (Statement) – The new statement

• position (int) – The optional position where to put the statement

Return type:

VariableReference | None

Returns: # noqa: DAR202

The return value of the statement. Notice that the test might choose to modify the statement you inserted. You should use the returned variable reference and not use references. Can be None, if this statement does not create a variable.

add_statements(statements)

Adds a list of statements to the end of the test case.

Parameters:

statements (list[Statement]) – The list of statements to add

Return type:

None

add_variable_creating_statement(statement, position=-1)

Overloaded version of add_statement that adds a statement.

Parameters:

• statement (VariableCreatingStatement) – The new statement

• position (int) – The optional position where to put the statement

Return type:

VariableReference

Returns: # noqa: DAR202

The return value of the statement. Notice that the test might choose to modify the statement you inserted. You should use the returned variable reference and not use references.

append_test_case(test_case)

Appends a test case to this test case.

Parameters:

test_case (TestCase) – The test case to append

Return type:

None

chop(pos)

Remove all statements after a given position.

Parameters:

pos (int) – The length of the test case after chopping

Return type:

None

clone(limit=None)

Provides a deep copy of the test case.

Parameters:

limit (Optional[int]) – Clone this test case only up to the given number of statements.

Returns:

DAR202

Return type:

A deep copy of this test case # noqa

contains(statement)

Determines whether or not the test case contains a specific statement.

Parameters:

statement (Statement) – The statement to search in the test case

Returns:

DAR202

Return type:

Whether or not the test case contains the statement # noqa

get_assertions()

Get all assertions that exist for this test case.

Return type:

list[Assertion]

get_dependencies(var)

Provides all variables on which var depends.

Parameters:

var (VariableReference) – the variable whose dependencies we are looking for.

Returns:

DAR202

Return type:

a set of variables on which var depends on. # noqa

get_forward_dependencies(var)

Provides all variables that depend on var.

Parameters:

var (VariableReference) – the variable for which we look for all dependent statements of

Returns:

DAR202

Return type:

a set of variables that depend on var. # noqa

get_statement(position)

Provides access to a statement at a given position.

Parameters:

position (int) – The position of the statement in the test case

Returns:

DAR202

Return type:

The statement at the position # noqa

has_statement(position)

Check if there is a statement at the given position.

Parameters:

position (int) – The index of the statement

Returns:

DAR202

Return type:

Whether or not there is a statement at the given position # noqa

remove(position)

Removes a statement a the given position.

Parameters:

position (int) – The position of the test case to be removed

Return type:

None

remove_statement(statement)

Remove the given statement from this test case.

Parameters:

statement (Statement) – The statement to remove.

Return type:

None

remove_statement_with_forward_dependencies(statement)

Removes the given statement along with all its forward dependencies.

Parameters:

statement (Statement) – The statement to remove

Return type:

list[int]

Returns:

A list of positions of statements that have been deleted

Raises:

ValueError – If the statement is not contained in the test case

remove_with_forward_dependencies(position)

Removes a statement at the given position along with all its forward dependencies.

Parameters:

position (int) – The position of the statement to remove

Return type:

list[int]

Returns:

A list of positions of statements that have been deleted

Raises:

ValueError – If the position is out of bounds for this test case

set_statement(statement, position)

Set new statement at position.

Parameters:

• statement (Statement) – the new statement

• position (int) – the position for the new statement

Returns:

DAR202

Return type:

A variable reference to the statements return value, if any # noqa

size()

Provides the number of statements in the test case.

Returns:

DAR202

Return type:

The number of statements in the test case # noqa

size_with_assertions()

Provides the number of statements and assertions in the test case.

Returns:

DAR202

Return type:

The number of statements and assertions in the test case # noqa

pynguin.ga.testcasefactory

Provides a factories for generating different kind of test cases.

class pynguin.ga.testcasefactory.RandomLengthTestCaseFactory(test_factory, test_cluster)

Create random test cases with random length.

get_test_case()

Retrieve a test case.

Returns:

DAR202

Return type:

A test case # noqa

class pynguin.ga.testcasefactory.SeededTestCaseFactory(delegate, population_provider)

Factory for getting seeded test cases.

With a certain probability a seeded testcase is returned instead of a randomly generated one. If a seeded testcase is returned, it is taken randomly from the pool of seeded testcases. If a randomly generated testcase is returned, the generation is delegated to the RandomLengthTestCaseFactory.

get_test_case()

Retrieve a test case.

Returns:

DAR202

Return type:

A test case # noqa

class pynguin.ga.testcasefactory.TestCaseFactory(test_factory)

Abstract class for test case factories.

abstract get_test_case()

Retrieve a test case.

Returns:

DAR202

Return type:

A test case # noqa

pynguin.testcase.execution

Contains all code related to test-case execution.

class pynguin.testcase.execution.AbstractTestCaseExecutor

Interface for a test case executor.

abstract add_observer(observer)

Add an execution observer.

Parameters:

observer (ExecutionObserver) – the observer to be added.

Return type:

None

abstract add_remote_observer(remote_observer)

Add a remote execution observer.

Parameters:

remote_observer (RemoteExecutionObserver) – the remote observer to be added.

Return type:

None

abstract clear_observers()

Remove all existing observers.

Return type:

None

abstract clear_remote_observers()

Remove all existing remote observers.

Return type:

None

abstract execute(test_case)

Executes all statements of the given test case.

Parameters:

test_case (TestCase) – the test case that should be executed.

Raises:

RuntimeError – If something goes wrong inside Pynguin during execution.

Return type:

ExecutionResult

Returns:

Result of the execution

execute_multiple(test_cases)

Executes multiple test cases.

Parameters:

test_cases (Iterable[TestCase]) – The test cases that should be executed.

Raises:

RuntimeError – If something goes wrong inside Pynguin during execution.

Yields:

The results of the execution

Return type:

Iterable[ExecutionResult]

abstract property module_provider: ModuleProvider

The module provider used by this executor.

Returns:

The used module provider

abstract property subject_properties: SubjectProperties

Provide access to the subject properties.

Returns:

The subject properties

abstract temporarily_add_observer(observer)

Temporarily add the given observer.

Parameters:

observer (ExecutionObserver) – The observer to add.

Return type:

AbstractContextManager[None]

abstract temporarily_add_remote_observer(remote_observer)

Temporarily add a remote observer.

Parameters:

remote_observer (RemoteExecutionObserver) – The remote observer to add.

Return type:

AbstractContextManager[None]

class pynguin.testcase.execution.ExecutionContext(module_provider)

Contains information required in the context of an execution.

The context contains, e.g., the used variables, modules, and the AST representation of the statements that should be executed.

add_new_module_alias(module_name, alias)

Add a new module alias.

Parameters:

• module_name (str) – The name of the module

• alias (str) – The alias

Return type:

None

get_reference_value(reference)

Resolve the given reference in this execution context.

Parameters:

reference (Reference) – The reference to resolve.

Raises:

ValueError – If the root of the reference can not be resolved.

Return type:

Any

Returns:

The value that is resolved.

property global_namespace: dict[str, ModuleType]

The global namespace.

Returns:

The global namespace

property local_namespace: dict[str, Any]

The local namespace.

Returns:

The local namespace

property module_aliases: NamingScope

The module aliases.

Returns:

A naming scope that maps the used modules to their alias.

node_for_assertion(assertion, statement_node)

Transforms the given assertion in an executable ast node.

Parameters:

• assertion (Assertion) – The assertion that should be converted.

• statement_node (stmt) – The ast node of the statement for the assertion.

Return type:

stmt

Returns:

An ast node.

node_for_statement(statement)

Transforms the given statement in an executable ast node.

Parameters:

statement (Statement) – The statement that should be converted.

Return type:

stmt

Returns:

An ast node.

replace_variable_value(variable, new_value)

Replace the value of the variable with the new value.

Parameters:

• variable (VariableReference) – The variable for which we want to replace the value

• new_value (Any) – The replacement value.

Return type:

None

property variable_names: NamingScope

The variable names.

Returns:

A naming scope that maps the used variables to their names.

static wrap_node_in_module(node)

Wraps the given node in a module, such that it can be executed.

Parameters:

node (stmt) – The node to wrap

Return type:

Module

Returns:

The module wrapping the nodes

class pynguin.testcase.execution.ExecutionObserver

An observer that can be used to observe the execution of a test case.

abstract after_remote_test_case_execution(test_case, result)

Called after test case execution from the main thread.

Note: This method is always called, though the data you expect in the execution might not be there, if the execution of the test case timed out. You are not allowed to access thread local state here (due to how threading.local works, it isn’t even possible ;)), but you can do some postprocessing with the data from the execution result here.

Parameters:

• test_case (TestCase) – The test cases that was executed

• result (ExecutionResult) – The execution result

Return type:

None

abstract before_remote_test_case_execution(test_case)

Called before test case execution from the main thread.

Note: This method can be called with several test cases before the after_remote_test_case_execution method is called.

Parameters:

test_case (TestCase) – The test cases that will be executed.

Return type:

None

abstract property remote_observer: RemoteExecutionObserver

The remote observer.

Returns:

The remote observer

class pynguin.testcase.execution.ExecutionResult(timeout=False)

Result of an execution.

delete_statement_data(deleted_statements)

Delete statements at given indices.

It may happen that the test case is modified after execution, for example, by removing unused primitives. We have to update the execution result to reflect this, otherwise the indexes maybe wrong.

Parameters:

deleted_statements (set[int]) – The indexes of the deleted statements

Return type:

None

get_first_position_of_thrown_exception()

Provide the index of the first thrown exception or None.

Return type:

int | None

Returns:

The index of the first thrown exception, if any

has_test_exceptions()

Returns true if any exceptions were thrown during the execution.

Return type:

bool

Returns:

Whether the test has exceptions

report_new_thrown_exception(stmt_idx, ex)

Report an exception that was thrown during execution.

Parameters:

• stmt_idx (int) – the index of the statement, that caused the exception

• ex (BaseException) – the exception

Return type:

None

static shift_dict(to_shift, deleted_indexes)

Shifts the entries in the given dictionary.

Compute the entries’ new positions after the given statements were deleted.

Parameters:

• to_shift (dict[int, TypeVar(T)]) – The dict to shift

• deleted_indexes (set[int]) – A set of deleted statement indexes.

Return type:

dict[int, TypeVar(T)]

Returns:

The shifted dict

class pynguin.testcase.execution.ModuleProvider

Class for providing modules.

add_mutated_version(module_name, mutated_module)

Adds a mutated version of a module to the collection of mutated modules.

Parameters:

• module_name (str) – for the module name of the module, which should be mutated.

• mutated_module (ModuleType) – the custom module, which should be used.

Return type:

None

clear_mutated_modules()

Clear the existing aliases.

get_module(module_name)

Provides a module.

Either from sys.modules or if a mutated version for the given module name exists then the mutated version of the module will be returned.

Parameters:

module_name (str) – string for the module alias, which should be loaded

Raises:

ModuleNotImportedError – If the module is not imported.

Return type:

ModuleType

Returns:

the module which should be loaded.

class pynguin.testcase.execution.OutputSuppressionContext

A context manager that suppresses stdout and stderr.

Operates at two levels: - Python level: redirects sys.stdout / sys.stderr to /dev/null. - OS level: saves file descriptors 0/1/2 via os.dup so that if the SUT

closes them (e.g. with open(1, 'w') where the int happens to be a stdio fd), they are restored on exit.

restore()

Restore stdout and stderr at both Python and OS level.

Return type:

None

class pynguin.testcase.execution.PatchRandomOnUnpickle

A hook that patches random when unpickled in a subprocess.

This ensures that random.Random.seed is patched before the SUT is unpickled and potentially creates new random.Random instances (e.g., mimesis).

class pynguin.testcase.execution.RemoteAssertionExecutionObserver

A remote observer which executes the assertions of statements.

Enables slicing on the recorded data.

after_statement_execution(statement, executor, exec_ctx, exception)

Called after a statement was executed.

Parameters:

• statement (Statement) – the statement that was executed.

• executor (TestCaseExecutor) – the executor, in case you want to execute something.

• exec_ctx (ExecutionContext) – the current execution context.

• exception (BaseException | None) – the exception that was thrown, if any.

Return type:

None

after_test_case_execution(executor, test_case, result)

Not used.

Parameters:

• executor (TestCaseExecutor) – Not used

• test_case (TestCase) – Not used

• result (ExecutionResult) – Not used

Return type:

None

before_statement_execution(statement, node, exec_ctx)

Called before a statement is executed.

Parameters:

• statement (Statement) – the statement about to be executed.

• node (stmt) – the ast node representing the statement.

• exec_ctx (ExecutionContext) – the current execution context.

Return type:

stmt

Returns:

An ast node. You may choose to modify this node to change what is executed.

before_test_case_execution(test_case)

Not used.

Parameters:

test_case (TestCase) – not used

class pynguin.testcase.execution.RemoteExecutionObserver

A remote observer that can be used to observe the execution of a test case.

Important Note: If an observer is stateful, then this state must be encapsulated in a threading.local, i.e., be bound to a thread. Note that thread local data is initialized per thread, so there is no need to clear any pre-existing data (because there is none), as every thread gets its own instance.

Methods in this class are not allowed to interact with the ‘outside’ because this class could be sent to a remote environment. The only thing that should leave an observer are results when they are written to the execution result in RemoteExecutionObserver::after_test_case_execution.

You may interact with the ‘outside’ in ExecutionObserver::after_remote_test_case_execution.

Note: Usage of threading.local may interfere with debugging tools, such as pydevd. In such a case, disable Cython by setting the following environment variable: PYDEVD_USE_CYTHON=NO

For more details, look at some implementations, e.g., AssertionTraceObserver.

abstract after_statement_execution(statement, executor, exec_ctx, exception)

Called after a statement was executed.

Parameters:

• statement (Statement) – the statement that was executed.

• executor (TestCaseExecutor) – the executor, in case you want to execute something.

• exec_ctx (ExecutionContext) – the current execution context.

• exception (BaseException | None) – the exception that was thrown, if any.

Return type:

None

abstract after_test_case_execution(executor, test_case, result)

Called after test case execution.

The call happens from the remote environment that executed the test case. You should override this method to extract information from the thread local storage to the execution result.

Note: When a timeout occurs, then this method might not be called at all.

Parameters:

• executor (TestCaseExecutor) – The executor that executed the test case

• test_case (TestCase) – The test cases that was executed

• result (ExecutionResult) – The execution result

Return type:

None

abstract before_statement_execution(statement, node, exec_ctx)

Called before a statement is executed.

Parameters:

• statement (Statement) – the statement about to be executed.

• node (stmt) – the ast node representing the statement.

• exec_ctx (ExecutionContext) – the current execution context.

Return type:

stmt

Returns:

An ast node. You may choose to modify this node to change what is executed.

abstract before_test_case_execution(test_case)

Called before test case execution.

Parameters:

test_case (TestCase) – The test cases that will be executed.

property state: dict[str, Any]

The state of the observer.

Returns:

The state of the observer

class pynguin.testcase.execution.RemoteReturnTypeObserver

An observer which observes the runtime types seen during execution.

class RemoteReturnTypeLocalState

Encapsulate observed return types.

after_statement_execution(statement, executor, exec_ctx, exception)

Called after a statement was executed.

Parameters:

• statement (Statement) – the statement that was executed.

• executor (TestCaseExecutor) – the executor, in case you want to execute something.

• exec_ctx (ExecutionContext) – the current execution context.

• exception (BaseException | None) – the exception that was thrown, if any.

Return type:

None

after_test_case_execution(executor, test_case, result)

Called after test case execution.

The call happens from the remote environment that executed the test case. You should override this method to extract information from the thread local storage to the execution result.

Note: When a timeout occurs, then this method might not be called at all.

Parameters:

• executor (TestCaseExecutor) – The executor that executed the test case

• test_case (TestCase) – The test cases that was executed

• result (ExecutionResult) – The execution result

before_statement_execution(statement, node, exec_ctx)

Called before a statement is executed.

Parameters:

• statement (Statement) – the statement about to be executed.

• node (stmt) – the ast node representing the statement.

• exec_ctx (ExecutionContext) – the current execution context.

Return type:

stmt

Returns:

An ast node. You may choose to modify this node to change what is executed.

before_test_case_execution(test_case)

Not used.

Parameters:

test_case (TestCase) – Not used

property state: dict[str, Any]

The state of the observer.

Returns:

The state of the observer

class pynguin.testcase.execution.RemoteTypeTracingObserver

A remote execution observer used for type tracing.

class RemoteTypeTracingLocalState

Thread local data for type tracing.

after_statement_execution(statement, executor, exec_ctx, exception=None)

Called after a statement was executed.

Parameters:

• statement (Statement) – the statement that was executed.

• executor (TestCaseExecutor) – the executor, in case you want to execute something.

• exec_ctx (ExecutionContext) – the current execution context.

• exception (Optional[BaseException]) – the exception that was thrown, if any.

Return type:

None

after_test_case_execution(executor, test_case, result)

Called after test case execution.

The call happens from the remote environment that executed the test case. You should override this method to extract information from the thread local storage to the execution result.

Note: When a timeout occurs, then this method might not be called at all.

Parameters:

• executor (TestCaseExecutor) – The executor that executed the test case

• test_case (TestCase) – The test cases that was executed

• result (ExecutionResult) – The execution result

Return type:

None

before_statement_execution(statement, node, exec_ctx)

Called before a statement is executed.

Parameters:

• statement (Statement) – the statement about to be executed.

• node (stmt) – the ast node representing the statement.

• exec_ctx (ExecutionContext) – the current execution context.

Return type:

stmt

Returns:

An ast node. You may choose to modify this node to change what is executed.

before_test_case_execution(test_case)

Not used.

Parameters:

test_case (TestCase) – Not used

property state: dict[str, Any]

The state of the observer.

Returns:

The state of the observer

class pynguin.testcase.execution.ReturnTypeObserver(test_cluster)

Observes the runtime types seen during execution.

Updates the return types of the called function with the observed types.

after_remote_test_case_execution(test_case, result)

Called after test case execution from the main thread.

Note: This method is always called, though the data you expect in the execution might not be there, if the execution of the test case timed out. You are not allowed to access thread local state here (due to how threading.local works, it isn’t even possible ;)), but you can do some postprocessing with the data from the execution result here.

Parameters:

• test_case (TestCase) – The test cases that was executed

• result (ExecutionResult) – The execution result

before_remote_test_case_execution(test_case)

Not used.

Parameters:

test_case (TestCase) – Not used

Return type:

None

property remote_observer: RemoteExecutionObserver

The remote observer.

Returns:

The remote observer

class pynguin.testcase.execution.SubprocessTestCaseExecutor(subject_properties, module_provider=None, maximum_test_execution_timeout=5, test_execution_time_per_statement=1)

An executor that executes the generated test cases in a subprocess.

class ConnectionStatus(value)

Status of the connection to the subprocess.

class SubprocessResultContext(test_cases_tuple, references_bindings, process, receiving_connection, connection_status, remote_observers)

Context for processing subprocess results.

execute(test_case)

Executes all statements of the given test case.

Parameters:

test_case (TestCase) – the test case that should be executed.

Raises:

RuntimeError – If something goes wrong inside Pynguin during execution.

Return type:

ExecutionResult

Returns:

Result of the execution

execute_multiple(test_cases)

Executes multiple test cases.

Parameters:

test_cases (Iterable[TestCase]) – The test cases that should be executed.

Raises:

RuntimeError – If something goes wrong inside Pynguin during execution.

Yields:

The results of the execution

Return type:

Iterable[ExecutionResult]

execute_with_exit_code(test_case)

Execute a test case in a subprocess and return the exit code.

This method executes a single test case in a separate subprocess and returns the exit code of the subprocess. If the subprocess crashes or times out, it returns None or a non-zero exit code.

Parameters:

test_case (TestCase) – The test case to execute

Return type:

int | None

Returns:

The exit code of the subprocess. A None or non-zero exit code indicates a crash.

class pynguin.testcase.execution.TestCaseExecutor(subject_properties, module_provider=None, maximum_test_execution_timeout=5, test_execution_time_per_statement=1)

An executor that executes the generated test cases.

add_observer(observer)

Add an execution observer.

Parameters:

observer (ExecutionObserver) – the observer to be added.

Return type:

None

add_remote_observer(remote_observer)

Add a remote execution observer.

Parameters:

remote_observer (RemoteExecutionObserver) – the remote observer to be added.

Return type:

None

clear_observers()

Remove all existing observers.

Return type:

None

clear_remote_observers()

Remove all existing remote observers.

Return type:

None

execute(test_case)

Executes all statements of the given test case.

Parameters:

test_case (TestCase) – the test case that should be executed.

Raises:

RuntimeError – If something goes wrong inside Pynguin during execution.

Return type:

ExecutionResult

Returns:

Result of the execution

execute_ast(ast_node, exec_ctx)

Execute the given ast_node in the given context.

You can use this in an observer if you also need to execute an AST Node.

Parameters:

• ast_node (Module) – The node to execute.

• exec_ctx (ExecutionContext) – The execution context

Return type:

BaseException | None

Returns:

The raised exception, if any.

property module_provider: ModuleProvider

The module provider used by this executor.

Returns:

The used module provider

set_instrument(instrument)

Set if the test is to be instrumented as well.

Parameters:

instrument (bool) – Whether to instrument the test and its assertions.

Return type:

None

property subject_properties: SubjectProperties

Provide access to the subject properties.

Returns:

The subject properties

temporarily_add_observer(observer)

Temporarily add the given observer.

Parameters:

observer (ExecutionObserver) – The observer to add.

Return type:

Generator[None, None, None]

temporarily_add_remote_observer(remote_observer)

Temporarily add a remote observer.

Parameters:

remote_observer (RemoteExecutionObserver) – The remote observer to add.

Return type:

Generator[None, None, None]

class pynguin.testcase.execution.TypeTracingObserver(cluster)

An execution observer used for type tracing.

It wraps parameters in proxies in order to make better guesses on their type.

after_remote_test_case_execution(test_case, result)

Called after test case execution from the main thread.

Note: This method is always called, though the data you expect in the execution might not be there, if the execution of the test case timed out. You are not allowed to access thread local state here (due to how threading.local works, it isn’t even possible ;)), but you can do some postprocessing with the data from the execution result here.

Parameters:

• test_case (TestCase) – The test cases that was executed

• result (ExecutionResult) – The execution result

Return type:

None

before_remote_test_case_execution(test_case)

Not used.

Parameters:

test_case (TestCase) – Not used

Return type:

None

property remote_observer: RemoteTypeTracingObserver

The remote observer.

Returns:

The remote observer

class pynguin.testcase.execution.TypeTracingTestCaseExecutor(delegate, cluster, type_tracing_probability=1.0)

A test case executor that delegates to another executor.

Every test case is executed twice, one time for the regular result and one time with proxies in order to refine parameter types.

add_observer(observer)

Add an execution observer.

Parameters:

observer (ExecutionObserver) – the observer to be added.

Return type:

None

add_remote_observer(remote_observer)

Add a remote execution observer.

Parameters:

remote_observer (RemoteExecutionObserver) – the remote observer to be added.

Return type:

None

clear_observers()

Remove all existing observers.

Return type:

None

clear_remote_observers()

Remove all existing remote observers.

Return type:

None

execute(test_case)

Executes all statements of the given test case.

Parameters:

test_case (TestCase) – the test case that should be executed.

Raises:

RuntimeError – If something goes wrong inside Pynguin during execution.

Return type:

ExecutionResult

Returns:

Result of the execution

property module_provider: ModuleProvider

The module provider used by this executor.

Returns:

The used module provider

property subject_properties: SubjectProperties

Provide access to the subject properties.

Returns:

The subject properties

temporarily_add_observer(observer)

Temporarily add the given observer.

Parameters:

observer (ExecutionObserver) – The observer to add.

Return type:

AbstractContextManager[None]

temporarily_add_remote_observer(remote_observer)

Temporarily add a remote observer.

Parameters:

remote_observer (RemoteExecutionObserver) – The remote observer to add.

Return type:

AbstractContextManager[None]

pynguin.instrumentation.tracer

Contains all code related to test-case execution.

class pynguin.instrumentation.tracer.AbstractExecutionTracer

An abstract execution tracer.

The results are stored in an execution trace.

abstract check()

Check if the thread that called this method should still be running.

Raises:

RuntimeError – if the thread is not running anymore.

Return type:

None

abstract disable()

Disable tracing.

Return type:

None

abstract enable()

Enable tracing.

Return type:

None

abstract executed_bool_predicate(value, predicate)

A predicate that is based on a boolean value was executed.

Parameters:

• value – the value

• predicate (int) – the predicate identifier

Raises:

RuntimeError – raised when called from another thread

Return type:

None

abstract executed_code_object(code_object_id)

Mark a code object as executed.

This means, that the routine which refers to this code object was at least called once.

Parameters:

code_object_id (int) – the code object id to mark

Raises:

RuntimeError – raised when called from another thread

Return type:

None

abstract executed_compare_predicate(value1, value2, predicate, cmp_op)

A predicate that is based on a comparison was executed.

Parameters:

• value1 – the first value

• value2 – the second value

• predicate (int) – the predicate identifier

• cmp_op (PynguinCompare) – the compare operation

Raises:

• RuntimeError – raised when called from another thread.

• AssertionError – when encountering an unknown compare op.

Return type:

None

abstract executed_exception_match(err, exc, predicate)

A predicate that is based on exception matching was executed.

Parameters:

• err (BaseException | type[BaseException]) – The raised exception

• exc (type[BaseException]) – The matching condition

• predicate (int) – the predicate identifier

Raises:

RuntimeError – raised when called from another thread

Return type:

None

abstract executed_in_presence_predicate(value1, value2, predicate)

An auxiliary membership predicate was executed.

Computes a guided branch distance for value1 in value2 with an execution-time guard on the container size to limit overhead.

Parameters:

• value1 – The prospective key/index to check membership for.

• value2 – The container to check membership in.

• predicate (int) – The predicate identifier.

Raises:

RuntimeError – raised when called from another thread

Return type:

None

abstract get_trace()

Get the trace with the current information.

Return type:

ExecutionTrace

Returns:

The current execution trace

abstract property import_trace: ExecutionTrace

The trace that was generated when the SUT was imported.

Returns:

The execution trace after executing the import statements

abstract init_trace()

Create a new trace that only contains the trace data from the import.

Return type:

None

abstract is_disabled()

Should we track anything?

We might have to disable tracing, e.g. when calling __eq__ ourselves. Otherwise, we create an endless recursion.

Return type:

bool

Returns:

Whether we should track anything

abstract reset()

Resets everything.

Should be called before instrumentation. Clears all data, so we can handle a reload of the SUT.

Return type:

None

abstract stop()

Stop the tracer.

This should be called when the tracer is no longer needed, e.g., when the test case execution is finished.

Return type:

None

abstract store_import_trace()

Stores the current trace as the import trace.

Should only be done once, after a module was loaded. The import trace will be merged into every subsequently recorded trace.

Return type:

None

temporarily_disable()

Temporarily disable tracing.

If the tracing is already disabled, do nothing.

Return type:

Generator[None, None, None]

temporarily_enable()

Temporarily enable tracing.

If the tracing is already enabled, do nothing.

Return type:

Generator[None, None, None]

abstract track_assertion_position(assertion)

Track the position of an assertion in the trace.

Parameters:

assertion (Assertion) – the assertion of the statement

Raises:

RuntimeError – raised when called from another thread

Return type:

None

abstract track_attribute_access(module, code_object_id, node_id, opcode, lineno, offset, attr_name, obj)

Track an attribute access instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• attr_name (str | None) – the name of the accessed attribute

• obj (object) – the object containing the accessed attribute

Raises:

RuntimeError – raised when called from another thread

Return type:

None

abstract track_call(module, code_object_id, node_id, opcode, lineno, offset, arg)

Track a method call instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• arg (int) – the argument used in the method call

Raises:

RuntimeError – raised when called from another thread

Return type:

None

abstract track_exception_assertion(statement)

Track the position of an exception assertion in the trace.

Normally, to track an assertion, we trace the POP_JUMP_IF_TRUE instruction contained by each assertion. The pytest exception assertion does not use an assertion containing this instruction. Therefore, we trace the instruction that was last executed before the exception.

Parameters:

statement (Statement) – the statement causing the exception

Raises:

RuntimeError – raised when called from another thread

Return type:

None

abstract track_generic(module, code_object_id, node_id, opcode, lineno, offset)

Track a generic instruction inside the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

Raises:

RuntimeError – raised when called from another thread

Return type:

None

abstract track_jump(module, code_object_id, node_id, opcode, lineno, offset, target_id)

Track a jump instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• target_id (int) – the offset of the target of the jump

Raises:

RuntimeError – raised when called from another thread

Return type:

None

abstract track_line_visit(line_id)

Tracks the visit of a line.

Parameters:

line_id (int) – the if of the line that was visited

Raises:

RuntimeError – raised when called from another thread

Return type:

None

abstract track_memory_access(module, code_object_id, node_id, opcode, lineno, offset, var_name, var_value)

Track a memory access instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• var_name (str | CellVar | FreeVar | tuple[str | CellVar | FreeVar, str | CellVar | FreeVar]) – the used variable name

• var_value (object) – the value stored in the used variable

Raises:

• ValueError – when no argument is given

• RuntimeError – raised when called from another thread

Return type:

None

abstract track_return(module, code_object_id, node_id, opcode, lineno, offset)

Track a return instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

Raises:

RuntimeError – raised when called from another thread

Return type:

None

class pynguin.instrumentation.tracer.CodeObjectMetaData(code_object, parent_code_object_id, cfg, cdg)

Stores meta data of a code object.

class pynguin.instrumentation.tracer.ExecutedAssertion(trace_position, assertion)

Data class for assertions of a testcase traced during execution for slicing.

class pynguin.instrumentation.tracer.ExecutionTrace(executed_code_objects=<factory>, executed_predicates=<factory>, true_distances=<factory>, false_distances=<factory>, covered_line_ids=<factory>, executed_instructions=<factory>, object_addresses=<factory>, executed_assertions=<factory>, checked_lines=<factory>)

Stores trace information about the execution.

add_attribute_instruction(module, code_object_id, node_id, opcode, lineno, offset, attr_name, src_address, arg_address, is_mutable_type, is_method)

Creates a new ExecutedAttributeInstruction object and adds it to the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• attr_name (str) – the name of the accessed attribute

• src_address (int) – the memory address of the attribute

• arg_address (int) – the memory address of the argument

• is_mutable_type (bool) – if the attribute is mutable

• is_method (bool) – if the attribute is a method

Return type:

None

add_call_instruction(module, code_object_id, node_id, opcode, lineno, offset, arg)

Creates a new ExecutedCallInstruction object and adds it to the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• arg (int) – the argument to the instruction

Return type:

None

add_instruction(module, code_object_id, node_id, opcode, lineno, offset)

Creates a new ExecutedInstruction object and adds it to the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

Return type:

None

add_jump_instruction(module, code_object_id, node_id, opcode, lineno, offset, target_id)

Creates a new ExecutedControlInstruction object and adds it to the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• target_id (int) – the target offset to jump to

Return type:

None

add_memory_instruction(module, code_object_id, node_id, opcode, lineno, offset, arg_name, arg_address, is_mutable_type, object_creation)

Creates a new ExecutedMemoryInstruction object and adds it to the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• arg_name (str | tuple[str, str]) – the name of the argument

• arg_address (int | tuple[int, int]) – the memory address of the argument

• is_mutable_type (bool | tuple[bool, bool]) – if the argument is mutable

• object_creation (bool | tuple[bool, bool]) – if the instruction creates the object used

Return type:

None

add_return_instruction(module, code_object_id, node_id, opcode, lineno, offset)

Creates a new ExecutedReturnInstruction object and adds it to the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

Return type:

None

merge(other)

Merge the values from the other execution trace.

Parameters:

other (ExecutionTrace) – Merges the other traces into this trace

Return type:

None

update_predicate_distances(distance_true, distance_false, predicate)

Update the distances and predicate execution count.

Parameters:

• distance_true (float) – the measured true distance

• distance_false (float) – the measured false distance

• predicate (int) – the predicate id

Return type:

None

class pynguin.instrumentation.tracer.ExecutionTracer

Tracks branch distances and covered statements during execution.

The results are stored in an execution trace.

class TracerLocalState

Encapsulate state that is thread specific.

static attribute_lookup(object_type, attribute)

Check the dictionary of classes making up the MRO (method resolution order).

It is inspired by the _PyType_Lookup C function in CPython.

Parameters:

• object_type – The type object to check

• attribute (str) – the attribute to check for in the class. It must be a data descriptor to be prioritized here.

Return type:

int

Returns:

The id of the object type or the class if it has the attribute, -1 otherwise

check()

Check if the thread that called this method should still be running.

Raises:

RuntimeError – if the thread is not running anymore.

Return type:

None

disable()

Disable tracing.

Return type:

None

enable()

Enable tracing.

Return type:

None

executed_bool_predicate(value, predicate)

A predicate that is based on a boolean value was executed.

Parameters:

• value – the value

• predicate (int) – the predicate identifier

Raises:

RuntimeError – raised when called from another thread

Return type:

None

executed_code_object(code_object_id)

Mark a code object as executed.

This means, that the routine which refers to this code object was at least called once.

Parameters:

code_object_id (int) – the code object id to mark

Raises:

RuntimeError – raised when called from another thread

Return type:

None

executed_compare_predicate(value1, value2, predicate, cmp_op)

A predicate that is based on a comparison was executed.

Parameters:

• value1 – the first value

• value2 – the second value

• predicate (int) – the predicate identifier

• cmp_op (PynguinCompare) – the compare operation

Raises:

• RuntimeError – raised when called from another thread.

• AssertionError – when encountering an unknown compare op.

Return type:

None

executed_exception_match(err, exc, predicate)

A predicate that is based on exception matching was executed.

Parameters:

• err (BaseException | type[BaseException]) – The raised exception

• exc (type[BaseException]) – The matching condition

• predicate (int) – the predicate identifier

Raises:

RuntimeError – raised when called from another thread

Return type:

None

executed_in_presence_predicate(value1, value2, predicate)

Conditionally compute an auxiliary ‘IN’ predicate distance.

This helper provides guidance for subscripts like container[key] by reporting a membership distance key in container before the subscript executes. To control overhead, it only computes a distance when the container is sized and its size does not exceed max_container_size.

Parameters:

• value1 – The prospective key/index to look up (e.g., key).

• value2 – The container to check membership in (e.g., container).

• predicate (int) – The predicate id to update.

Return type:

None

get_trace()

Get the trace with the current information.

Return type:

ExecutionTrace

Returns:

The current execution trace

property import_trace: ExecutionTrace

The trace that was generated when the SUT was imported.

Returns:

The execution trace after executing the import statements

init_trace()

Create a new trace that only contains the trace data from the import.

Return type:

None

is_disabled()

Should we track anything?

We might have to disable tracing, e.g. when calling __eq__ ourselves. Otherwise, we create an endless recursion.

Return type:

bool

Returns:

Whether we should track anything

reset()

Resets everything.

Should be called before instrumentation. Clears all data, so we can handle a reload of the SUT.

Return type:

None

property state: dict

Get the current state.

Returns:

The current state

stop()

Stop the tracer.

This should be called when the tracer is no longer needed, e.g., when the test case execution is finished.

Return type:

None

store_import_trace()

Stores the current trace as the import trace.

Should only be done once, after a module was loaded. The import trace will be merged into every subsequently recorded trace.

Return type:

None

track_assertion_position(assertion)

Track the position of an assertion in the trace.

Parameters:

assertion (Assertion) – the assertion of the statement

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_attribute_access(module, code_object_id, node_id, opcode, lineno, offset, attr_name, obj)

Track an attribute access instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• attr_name (str | None) – the name of the accessed attribute

• obj (object) – the object containing the accessed attribute

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_call(module, code_object_id, node_id, opcode, lineno, offset, arg)

Track a method call instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• arg (int) – the argument used in the method call

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_exception_assertion(statement)

Track the position of an exception assertion in the trace.

Normally, to track an assertion, we trace the POP_JUMP_IF_TRUE instruction contained by each assertion. The pytest exception assertion does not use an assertion containing this instruction. Therefore, we trace the instruction that was last executed before the exception.

Parameters:

statement (Statement) – the statement causing the exception

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_generic(module, code_object_id, node_id, opcode, lineno, offset)

Track a generic instruction inside the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_jump(module, code_object_id, node_id, opcode, lineno, offset, target_id)

Track a jump instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• target_id (int) – the offset of the target of the jump

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_line_visit(line_id)

Tracks the visit of a line.

Parameters:

line_id (int) – the if of the line that was visited

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_memory_access(module, code_object_id, node_id, opcode, lineno, offset, var_name, var_value)

Track a memory access instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• var_name (str | CellVar | FreeVar | tuple[str | CellVar | FreeVar, str | CellVar | FreeVar]) – the used variable name

• var_value (object) – the value stored in the used variable

Raises:

• ValueError – when no argument is given

• RuntimeError – raised when called from another thread

Return type:

None

track_return(module, code_object_id, node_id, opcode, lineno, offset)

Track a return instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

Raises:

RuntimeError – raised when called from another thread

Return type:

None

class pynguin.instrumentation.tracer.InstrumentationExecutionTracer(tracer)

An InstrumentationExecutionTracer is a sort of proxy for an ExecutionTracer.

This was done because when a module is instrumented, instructions are inserted into its bytecode and refer directly to a tracer. This means that without the use of a proxy, it would be impossible to modify the tracer, as there are direct references between the bytecode instructions and the tracer. By adding a proxy between the bytecode and the tracer, this ensures that the bytecode only has direct references to the proxy but no references to the tracer, so the tracer can be modified without any problems.

check()

Check if the thread that called this method should still be running.

Raises:

RuntimeError – if the thread is not running anymore.

Return type:

None

disable()

Disable tracing.

Return type:

None

enable()

Enable tracing.

Return type:

None

executed_bool_predicate(value, predicate)

A predicate that is based on a boolean value was executed.

Parameters:

• value – the value

• predicate (int) – the predicate identifier

Raises:

RuntimeError – raised when called from another thread

Return type:

None

executed_code_object(code_object_id)

Mark a code object as executed.

This means, that the routine which refers to this code object was at least called once.

Parameters:

code_object_id (int) – the code object id to mark

Raises:

RuntimeError – raised when called from another thread

Return type:

None

executed_compare_predicate(value1, value2, predicate, cmp_op)

A predicate that is based on a comparison was executed.

Parameters:

• value1 – the first value

• value2 – the second value

• predicate (int) – the predicate identifier

• cmp_op (PynguinCompare) – the compare operation

Raises:

• RuntimeError – raised when called from another thread.

• AssertionError – when encountering an unknown compare op.

Return type:

None

executed_exception_match(err, exc, predicate)

A predicate that is based on exception matching was executed.

Parameters:

• err (BaseException | type[BaseException]) – The raised exception

• exc (type[BaseException]) – The matching condition

• predicate (int) – the predicate identifier

Raises:

RuntimeError – raised when called from another thread

Return type:

None

executed_in_presence_predicate(value1, value2, predicate)

An auxiliary membership predicate was executed.

Computes a guided branch distance for value1 in value2 with an execution-time guard on the container size to limit overhead.

Parameters:

• value1 – The prospective key/index to check membership for.

• value2 – The container to check membership in.

• predicate (int) – The predicate identifier.

Raises:

RuntimeError – raised when called from another thread

Return type:

None

get_trace()

Get the trace with the current information.

Return type:

ExecutionTrace

Returns:

The current execution trace

property import_trace: ExecutionTrace

The trace that was generated when the SUT was imported.

Returns:

The execution trace after executing the import statements

init_trace()

Create a new trace that only contains the trace data from the import.

Return type:

None

is_disabled()

Should we track anything?

We might have to disable tracing, e.g. when calling __eq__ ourselves. Otherwise, we create an endless recursion.

Return type:

bool

Returns:

Whether we should track anything

reset()

Resets everything.

Should be called before instrumentation. Clears all data, so we can handle a reload of the SUT.

Return type:

None

stop()

Stop the tracer.

This should be called when the tracer is no longer needed, e.g., when the test case execution is finished.

Return type:

None

store_import_trace()

Stores the current trace as the import trace.

Should only be done once, after a module was loaded. The import trace will be merged into every subsequently recorded trace.

Return type:

None

track_assertion_position(assertion)

Track the position of an assertion in the trace.

Parameters:

assertion (Assertion) – the assertion of the statement

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_attribute_access(module, code_object_id, node_id, opcode, lineno, offset, attr_name, obj)

Track an attribute access instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• attr_name (str | None) – the name of the accessed attribute

• obj (object) – the object containing the accessed attribute

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_call(module, code_object_id, node_id, opcode, lineno, offset, arg)

Track a method call instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• arg (int) – the argument used in the method call

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_exception_assertion(statement)

Track the position of an exception assertion in the trace.

Normally, to track an assertion, we trace the POP_JUMP_IF_TRUE instruction contained by each assertion. The pytest exception assertion does not use an assertion containing this instruction. Therefore, we trace the instruction that was last executed before the exception.

Parameters:

statement (Statement) – the statement causing the exception

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_generic(module, code_object_id, node_id, opcode, lineno, offset)

Track a generic instruction inside the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_jump(module, code_object_id, node_id, opcode, lineno, offset, target_id)

Track a jump instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• target_id (int) – the offset of the target of the jump

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_line_visit(line_id)

Tracks the visit of a line.

Parameters:

line_id (int) – the if of the line that was visited

Raises:

RuntimeError – raised when called from another thread

Return type:

None

track_memory_access(module, code_object_id, node_id, opcode, lineno, offset, var_name, var_value)

Track a memory access instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

• var_name (str | CellVar | FreeVar | tuple[str | CellVar | FreeVar, str | CellVar | FreeVar]) – the used variable name

• var_value (object) – the value stored in the used variable

Raises:

• ValueError – when no argument is given

• RuntimeError – raised when called from another thread

Return type:

None

track_return(module, code_object_id, node_id, opcode, lineno, offset)

Track a return instruction in the trace.

Parameters:

• module (str) – File name of the module containing the instruction

• code_object_id (int) – code object containing the instruction

• node_id (int) – the node of the code object containing the instruction

• opcode (int) – the opcode of the instruction

• lineno (int) – the line number of the instruction

• offset (int) – the offset of the instruction

Raises:

RuntimeError – raised when called from another thread

Return type:

None

class pynguin.instrumentation.tracer.LineMetaData(code_object_id, file_name, line_number)

Stores meta data of a line.

class pynguin.instrumentation.tracer.PredicateMetaData(line_no, code_object_id, node)

Stores meta data of a predicate.

class pynguin.instrumentation.tracer.SubjectProperties(instrumentation_tracer=<factory>, code_object_counter=<factory>, existing_code_objects=<factory>, existing_predicates=<factory>, existing_lines=<factory>)

Contains properties about the subject under test.

The subject properties are code objects, predicates and lines:

• Code Objects: Compiled chunks of code (functions, methods, modules). Tracked in CodeObjectMetaData with references to the compiled code, parent, control graphs, and a unique ID. Represent the program’s structural units.

• Predicates: Decision points within code objects (e.g., if, while). Tracked in PredicateMetaData with line number, owning code object, and graph node. Used for branch coverage and measuring branch distances.

• Lines: Individual lines of code within code objects. Tracked in LineMetaData with file name and line number. Used for measuring line coverage.

Example:

def example(x):
    if x > 0:  # Predicate
        return "pos"
    return "non-pos"

# The function ``example`` is a Code Object and ``x > 0`` is a Predicate.

property branch_less_code_objects: Iterable[int]

Get the existing code objects that do not contain a branch.

Every code object is initially seen as branch-less until a predicate is registered for it.

Returns:

The existing code objects that do not contain a branch.

create_code_object_id()

Create a new code object ID.

Return type:

int

Returns:

A new code object ID.

lineids_to_linenos(line_ids)

Convenience method to translate line ids to line numbers.

Parameters:

line_ids (OrderedSet[int]) – The ids that should be translated.

Return type:

OrderedSet[int]

Returns:

The line numbers.

register_code_object(code_object_id, meta)

Declare that a code object exists.

Parameters:

• code_object_id (int) – the id of the code object, which should be used to identify the object

• instrumentation. (during)

• meta (CodeObjectMetaData) – the code objects existing

Return type:

None

register_line(meta)

Tracks the existence of a line.

Parameters:

meta (LineMetaData) – Metadata about the line

Return type:

int

Returns:

the id of the registered line

register_predicate(meta)

Declare that a predicate exists.

Parameters:

meta (PredicateMetaData) – Metadata about the predicates

Return type:

int

Returns:

the id of the predicate, which can be used to identify the predicate during instrumentation.

reset()

Resets the subject properties.

Return type:

None

validate_execution_trace(execution_trace)

Validate the execution trace.

Parameters:

execution_trace (ExecutionTrace) – The execution trace to validate

Raises:

AssertionError – if the execution trace is invalid

Return type:

None

pynguin.ga.computations

Provides classes for computations on chromosomes, e.g., fitness and coverage.

class pynguin.ga.computations.BranchDistanceTestCaseFitnessFunction(executor, code_object_id)

A fitness function based on branch distances and entered code objects.

compute_fitness(individual)

Calculate the fitness value.

Parameters:

individual – the chromosome to compute the fitness for.

Returns:

DAR202

Return type:

the new fitness # noqa

compute_is_covered(individual)

Compute if the goal of this fitness function is covered.

This computation is usually cheaper than computing the fitness, because we are not interested in the distance, but only a boolean result.

Parameters:

individual – the chromosome to check coverage on.

Return type:

bool

Returns:

True, if the goal of this fitness function is covered.

is_maximisation_function()

Do we need to maximise or minimise this function?

Returns:

DAR202

Return type:

Whether or not this is a maximisation function # noqa

class pynguin.ga.computations.BranchDistanceTestSuiteFitnessFunction(executor)

A fitness function based on branch distances and entered code objects.

compute_fitness(individual)

Calculate the fitness value.

Parameters:

individual – the chromosome to compute the fitness for.

Returns:

DAR202

Return type:

the new fitness # noqa

compute_is_covered(individual)

Compute if the goal of this fitness function is covered.

This computation is usually cheaper than computing the fitness, because we are not interested in the distance, but only a boolean result.

Parameters:

individual – the chromosome to check coverage on.

Return type:

bool

Returns:

True, if the goal of this fitness function is covered.

is_maximisation_function()

Do we need to maximise or minimise this function?

Returns:

DAR202

Return type:

Whether or not this is a maximisation function # noqa

restrict(exclude_code, exclude_true, exclude_false)

Restrict this fitness function.

Restricts the fitness function with respect to the branches/code objects it considers.

Parameters:

• exclude_code (set[int]) – Ids of the code objects that should not be considered.

• exclude_true (set[int]) – Ids of predicates whose True branch should not be considered.

• exclude_false (set[int]) – Ids of predicates whose False branch should not be considered.

Return type:

None

class pynguin.ga.computations.ChromosomeComputation(_executor)

An abstract computation on chromosomes.

class pynguin.ga.computations.ComputationCache(chromosome, *, fitness_functions=None, coverage_functions=None, fitness_cache=None, is_covered_cache=None, coverage_cache=None)

Caches computation results and computes values on demand.

add_coverage_function(coverage_function)

Adds a coverage function.

Parameters:

coverage_function (CoverageFunction) – A fitness function

Return type:

None

add_fitness_function(fitness_function)

Adds the given fitness function.

Parameters:

fitness_function (FitnessFunction) – A fitness function

Return type:

None

clone(new_chromosome)

Create a deep copy of this cache.

Parameters:

new_chromosome – The chromosome with which this cache is associated.

Return type:

ComputationCache

Returns:

A deep copy.

get_coverage()

Provides the mean coverage value.

Return type:

float

Returns:

The mean coverage value

get_coverage_for(coverage_function)

Provides the coverage value for a certain coverage function.

Parameters:

coverage_function (CoverageFunction) – The fitness function whose coverage value shall be returned

Return type:

float

Returns:

The coverage value for the fitness function

get_coverage_functions()

Provide the currently configured coverage functions of this chromosome.

Return type:

list[CoverageFunction]

Returns:

The list of currently configured coverage functions.

get_fitness()

Provide a sum of the current fitness values.

Return type:

float

Returns:

The sum of the current fitness values

get_fitness_for(fitness_function)

Returns the fitness values of a specific fitness function.

Parameters:

fitness_function (FitnessFunction) – The fitness function

Return type:

float

Returns:

Its fitness value

get_fitness_functions()

Provide the currently configured fitness functions of this chromosome.

Return type:

list[FitnessFunction]

Returns:

The list of currently configured fitness functions

get_is_covered(fitness_function)

Check if the individual covers this fitness function.

Parameters:

fitness_function (FitnessFunction) – The fitness function to check

Return type:

bool

Returns:

True, iff the individual covers the fitness function.

invalidate_cache()

Invalidate all cached computation values.

Return type:

None

set_coverage_values(coverage_values)

Sets the coverage values for the specific functions.

Parameters:

coverage_values (dict[CoverageFunction, float]) – A dictionary of coverage values, keyed by coverage function.

Return type:

None

set_fitness_values(fitness_values)

Sets the fitness values for the specific functions.

Parameters:

fitness_values (dict[FitnessFunction, float]) – A dictionary of fitness values, keyed by fitness function.

Return type:

None

class pynguin.ga.computations.CoverageFunction

Interface for a coverage function.

abstract compute_coverage(individual)

Compute the coverage of the given individual.

Parameters:

individual – the chromosome to compute the coverage for.

Return type:

float

Returns:

The computed coverage.

class pynguin.ga.computations.FitnessFunction

Interface for a fitness function.

abstract compute_fitness(individual)

Calculate the fitness value.

Parameters:

individual – the chromosome to compute the fitness for.

Returns:

DAR202

Return type:

the new fitness # noqa

abstract compute_is_covered(individual)

Compute if the goal of this fitness function is covered.

This computation is usually cheaper than computing the fitness, because we are not interested in the distance, but only a boolean result.

Parameters:

individual – the chromosome to check coverage on.

Return type:

bool

Returns:

True, if the goal of this fitness function is covered.

abstract is_maximisation_function()

Do we need to maximise or minimise this function?

Returns:

DAR202

Return type:

Whether or not this is a maximisation function # noqa

class pynguin.ga.computations.LineTestSuiteFitnessFunction(_executor)

A fitness function based on lines covered and entered code objects.

compute_fitness(individual)

Calculate the fitness value.

Parameters:

individual – the chromosome to compute the fitness for.

Returns:

DAR202

Return type:

the new fitness # noqa

compute_is_covered(individual)

Compute if the goal of this fitness function is covered.

This computation is usually cheaper than computing the fitness, because we are not interested in the distance, but only a boolean result.

Parameters:

individual – the chromosome to check coverage on.

Return type:

bool

Returns:

True, if the goal of this fitness function is covered.

is_maximisation_function()

Do we need to maximise or minimise this function?

Returns:

DAR202

Return type:

Whether or not this is a maximisation function # noqa

class pynguin.ga.computations.StatementCheckedTestSuiteFitnessFunction(_executor)

A fitness function for the checked statement coverage of test suites.

A fitness function based on lines included in the backward slice of each statement of a test suite.

compute_fitness(individual)

Calculate the fitness value.

Parameters:

individual – the chromosome to compute the fitness for.

Returns:

DAR202

Return type:

the new fitness # noqa

compute_is_covered(individual)

Compute if the goal of this fitness function is covered.

This computation is usually cheaper than computing the fitness, because we are not interested in the distance, but only a boolean result.

Parameters:

individual – the chromosome to check coverage on.

Return type:

bool

Returns:

True, if the goal of this fitness function is covered.

is_maximisation_function()

Do we need to maximise or minimise this function?

Returns:

DAR202

Return type:

Whether or not this is a maximisation function # noqa

class pynguin.ga.computations.TestCaseAssertionCheckedCoverageFunction(_executor)

Computes checked coverage on test cases with assertions.

compute_coverage(individual)

Compute the coverage of the given individual.

Parameters:

individual – the chromosome to compute the coverage for.

Return type:

float

Returns:

The computed coverage.

class pynguin.ga.computations.TestCaseBranchCoverageFunction(_executor)

Computes branch coverage on test cases.

compute_coverage(individual)

Compute the coverage of the given individual.

Parameters:

individual – the chromosome to compute the coverage for.

Return type:

float

Returns:

The computed coverage.

class pynguin.ga.computations.TestCaseChromosomeComputation(_executor)

A function that computes something on a test case chromosome.

class pynguin.ga.computations.TestCaseCoverageFunction(_executor)

Base class for all coverage functions that act on test case level.

class pynguin.ga.computations.TestCaseFitnessFunction(executor, code_object_id)

Base class for test case fitness functions.

property code_object_id: int

The code object id, where the target of the fitness function is located.

Returns:

The code object id where the target of the fitness function is located.

class pynguin.ga.computations.TestCaseLineCoverageFunction(_executor)

Computes line coverage on test cases.

compute_coverage(individual)

Compute the coverage of the given individual.

Parameters:

individual – the chromosome to compute the coverage for.

Return type:

float

Returns:

The computed coverage.

class pynguin.ga.computations.TestCaseStatementCheckedCoverageFunction(_executor)

Computes checked coverage on the statements of test cases.

compute_coverage(individual)

Compute the coverage of the given individual.

Parameters:

individual – the chromosome to compute the coverage for.

Return type:

float

Returns:

The computed coverage.

class pynguin.ga.computations.TestSuiteAssertionCheckedCoverageFunction(_executor)

Computes checked coverage on test suites with assertions.

compute_coverage(individual)

Compute the coverage of the given individual.

Parameters:

individual – the chromosome to compute the coverage for.

Return type:

float

Returns:

The computed coverage.

class pynguin.ga.computations.TestSuiteBranchCoverageFunction(_executor)

Computes branch coverage on test suites.

compute_coverage(individual)

Compute the coverage of the given individual.

Parameters:

individual – the chromosome to compute the coverage for.

Return type:

float

Returns:

The computed coverage.

class pynguin.ga.computations.TestSuiteChromosomeComputation(_executor)

A function that computes something on a test suite chromosome.

class pynguin.ga.computations.TestSuiteCoverageFunction(_executor)

Base class for all coverage functions that act on test suite level.

class pynguin.ga.computations.TestSuiteFitnessFunction(_executor)

Base class for test suite fitness functions.

class pynguin.ga.computations.TestSuiteLineCoverageFunction(_executor)

Computes line coverage on test suites.

compute_coverage(individual)

Compute the coverage of the given individual.

Parameters:

individual – the chromosome to compute the coverage for.

Return type:

float

Returns:

The computed coverage.

class pynguin.ga.computations.TestSuiteStatementCheckedCoverageFunction(_executor)

Computes checked coverage on the statements of test suites.

compute_coverage(individual)

Compute the coverage of the given individual.

Parameters:

individual – the chromosome to compute the coverage for.

Return type:

float

Returns:

The computed coverage.

pynguin.ga.computations.analyze_results(results)

Merge the trace of the given results.

Parameters:

results (list[ExecutionResult]) – The list of execution results to analyze

Return type:

ExecutionTrace

Returns:

the merged traces.

pynguin.ga.computations.compare(fitness_1, fitness_2)

Compare the two specified values.

Parameters:

• fitness_1 (float) – The first value to compare

• fitness_2 (float) – The second value to compare

Return type:

int

Returns:

the value 0 if fitness_1 is equal to fitness_2; a value less than 0 if fitness_1 is less than fitness_2; and a value greater than 0 if fitness_1 is greater than fitness_2

pynguin.ga.computations.compute_assertion_checked_coverage(trace, subject_properties)

Computes checked coverage on bytecode instructions.

Each assertion can be sliced, returning a list of instructions that are checked by an assertion. If we combine all lists of instructions returned by slicing all assertions, we get the combined dynamic slice of the test execution’s assertions. We then can map all instructions inside the slice to lines that are checked covered of the module under test. To calculate the coverage we can then divide the amount of lines checked covered through the test execution by the lines overall available in the module under test.

Parameters:

• trace (ExecutionTrace) – The execution trace

• subject_properties (SubjectProperties) – All known data

Return type:

float

Returns:

The computed coverage value

pynguin.ga.computations.compute_branch_coverage(trace, subject_properties)

Computes branch coverage on bytecode instructions.

The resulting coverage should be equal to decision coverage on source code.

Parameters:

• trace (ExecutionTrace) – The execution trace

• subject_properties (SubjectProperties) – All known data

Return type:

float

Returns:

The computed coverage value

pynguin.ga.computations.compute_branch_distance_fitness(trace, subject_properties, exclude_code=None, exclude_true=None, exclude_false=None)

Computes fitness based on covered branches and branch distances.

Parameters:

• trace (ExecutionTrace) – The execution trace

• subject_properties (SubjectProperties) – All known data

• exclude_code (Optional[set[int]]) – Ids of the code objects that should not be considered.

• exclude_true (Optional[set[int]]) – Ids of predicates whose True branch should not be considered.

• exclude_false (Optional[set[int]]) – Ids of predicates whose False branch should not be considered.

Return type:

float

Returns:

The computed fitness value

pynguin.ga.computations.compute_branch_distance_fitness_is_covered(trace, subject_properties, exclude_code=None, exclude_true=None, exclude_false=None)

Computes if all branches and code objects have been executed.

Parameters:

• trace (ExecutionTrace) – The execution trace

• subject_properties (SubjectProperties) – All known data

• exclude_code (Optional[set[int]]) – Ids of the code objects that should not be considered.

• exclude_true (Optional[set[int]]) – Ids of predicates whose True branch should not be considered.

• exclude_false (Optional[set[int]]) – Ids of predicates whose False branch should not be considered.

Return type:

bool

Returns:

True, if all branches were covered

pynguin.ga.computations.compute_checked_coverage_statement_fitness_is_covered(trace, subject_properties)

Computes if all lines and code objects are checked by a return statement.

Parameters:

• trace (ExecutionTrace) – The execution trace

• subject_properties (SubjectProperties) – All known data

Return type:

bool

Returns:

True, if all lines were checked by a return, false otherwise

pynguin.ga.computations.compute_line_coverage(trace, subject_properties)

Computes line coverage on bytecode instructions.

Parameters:

• trace (ExecutionTrace) – The execution trace

• subject_properties (SubjectProperties) – All known data

Return type:

float

Returns:

The computed coverage value

pynguin.ga.computations.compute_line_coverage_fitness_is_covered(trace, subject_properties)

Computes if all lines and code objects have been executed.

Parameters:

• trace (ExecutionTrace) – The execution trace

• subject_properties (SubjectProperties) – All known data

Return type:

bool

Returns:

True, if all lines were covered, false otherwise

pynguin.ga.computations.compute_statement_checked_lines(statements, trace, subject_properties, statement_slicing_criteria)

Computes checked coverage on bytecode instructions.

Each statement can be sliced, returning a list of instructions that are checked by the return value of the statement. If we combine all lists of instructions returned by slicing all statements, we get the combined dynamic slice of the test execution’s statements. We then can map all instructions inside the slice to lines that are checked covered of the module under test.

Parameters:

• statements (list[Statement]) – The sliced instructions

• trace (ExecutionTrace) – The execution trace

• subject_properties (SubjectProperties) – All known data

• statement_slicing_criteria (dict[int, SlicingCriterion]) – a dictionary of statement positions and its slicing criteria

Return type:

set[int]

Returns:

The checked line ids of lines checked by the statements

pynguin.ga.computations.normalise(value)

Normalise a value.

Parameters:

value (float) – The value to normalise

Return type:

float

Returns:

The normalised value

Raises:

RuntimeError – if the value is negative

pynguin.assertion.assertiongenerator

Provides an assertion generator.

class pynguin.assertion.assertiongenerator.AssertionGenerator(plain_executor, filtering_executions=1)

A simple assertion generator.

Creates all regression assertions.

visit_test_case_chromosome(chromosome)

Visit a test suite chromosome.

Parameters:

chromosome (TestCaseChromosome) – The test case chromosome

Return type:

None

visit_test_suite_chromosome(chromosome)

Visit a test suite chromosome.

Parameters:

chromosome (TestSuiteChromosome) – The test suite chromosome

Return type:

None

class pynguin.assertion.assertiongenerator.MutationAnalysisAssertionGenerator(plain_executor, mutation_controller, *, testing=False)

Uses mutation analysis to filter out less relevant assertions.

pynguin.assertion.mutation_analysis.controller

Provides a controller for generating mutants.

class pynguin.assertion.mutation_analysis.controller.MutationController(mutant_generator, module_ast, module)

A controller that creates mutants.

create_mutant(mutant_ast)

Creates a mutant of the module.

Parameters:

mutant_ast (Module) – The mutant AST.

Return type:

ModuleType

Returns:

The created mutant module.

create_mutants()

Creates mutants for the module.

Return type:

Generator[tuple[ModuleType | None, list[Mutation]]]

Returns:

A generator of tuples where the first entry is the mutated module or None if the mutated module cannot be created and the second part is a list of all the mutations operators applied.

mutant_count()

Calculates the number of mutants that can be created.

Return type:

int

Returns:

The number of mutants that can be created.

pynguin.assertion.mutation_analysis.mutators

Provides classes for mutating ASTs.

Based on https://github.com/se2p/mutpy-pynguin/blob/main/mutpy/controller.py and integrated in Pynguin.

class pynguin.assertion.mutation_analysis.mutators.FirstOrderMutator(operators)

A mutator that applies first order mutations.

mutate(target_ast, module)

Mutate the given AST.

Parameters:

• target_ast (AST) – The AST to mutate.

• module (ModuleType) – The module to mutate.

Yields:

A generator of mutations and the mutated AST.

Return type:

Generator[tuple[list[Mutation], AST]]

class pynguin.assertion.mutation_analysis.mutators.HighOrderMutator(operators, hom_strategy=None)

A mutator that applies high order mutations.

mutate(target_ast, module)

Mutate the given AST.

Parameters:

• target_ast (AST) – The AST to mutate.

• module (ModuleType) – The module to mutate.

Yields:

A generator of mutations and the mutated AST.

Return type:

Generator[tuple[list[Mutation], AST]]

class pynguin.assertion.mutation_analysis.mutators.Mutator

A mutator is responsible for mutating an AST.

abstract mutate(target_ast, module)

Mutate the given AST.

Parameters:

• target_ast (AST) – The AST to mutate.

• module (ModuleType) – The module to mutate.

Yields:

A generator of mutations and the mutated AST.

Return type:

Generator[tuple[list[Mutation], AST]]

pynguin.instrumentation.version.DynamicSeedingInstrumentation

Specialized instrumentation adapter for dynamic constant seeding in Python 3.10.

pynguin.utils.typetracing

Provides utilities to trace the usage of objects.

class pynguin.utils.typetracing.DepthDefaultDict(depth)

A dictionary creating a UsageTraceNode automatically for each key.

The implementation creates a UsageTraceNode for each requested and non-existing key.

class pynguin.utils.typetracing.UsageTraceNode(name, depth=0, type_checks=<factory>, arg_types=<factory>, arg_values=<factory>)

The knowledge gathered by a proxy.

find_path(path)

Check if this usage trace tree has the given path.

Parameters:

path (tuple[str, ...]) – The path to check

Return type:

UsageTraceNode | None

Returns:

The usage trace node at the end of the path, if it exists, otherwise None.

static from_proxy(obj)

Extract knowledge from the given proxy.

This is a convenience method, because the knowledge attribute is not visible on a proxy.

Parameters:

obj (ObjectProxy) – the proxy from which we should extract knowledge

Return type:

UsageTraceNode

Returns:

The extracted knowledge.

merge(other)

Merge the knowledge from the other proxy into this one.

Parameters:

other (UsageTraceNode) – The knowledge that should be merged into this one.

Return type:

None

pretty()

Create a pretty representation of this object.

Return type:

str

Returns:

A nicely formatted string

pynguin.utils.typetracing.proxify(*, log_args=False, no_wrap_return=False)

Decorator method to trace the usage of a method on a proxy.

1. Unwraps the proxy.

2. Stores the access to the method

3. Stores the argument types if requested.

4. Stores the argument values if requested and if the type is in VALUE_TRACED_TYPES.

5. Wraps the result in a proxy object (unless requested otherwise).

Parameters:

• log_args – Should we store the arguments (types and values)?

• no_wrap_return – Some cases, e.g., __int__ don’t allow a return value that is not an int, so in some cases we have to disable wrapping.

Returns:

A decorated function

pynguin.utils.typetracing.shim_isinstance()

Context manager that temporarily replaces isinstance with a shim.

The shim is aware of ObjectProxies.

Yields:

resets the shim

pynguin.utils.typetracing.unwrap(obj)

Unwrap the given object if it is a Proxy.

Parameters:

obj – The object to unwrap

Returns:

The unwrapped object