# pointblank

> Find out if your data is what you think it is.

## Docs

### API Reference

#### Validate
> When performing data validation, use the `Validate` class to get the process started. It takes the target table and options for metadata and failure thresholds (using the `Thresholds` class or shorthands). The `Validate` class has numerous methods for defining validation steps and for obtaining post-interrogation metrics and data.


- [Validate](reference/Validate.html): Workflow for defining a set of validations on a table and interrogating for results
- [Thresholds](reference/Thresholds.html): Definition of threshold values
- [Actions](reference/Actions.html): Definition of action values
- [FinalActions](reference/FinalActions.html): Define actions to be taken after validation is complete
- [Schema](reference/Schema.html): Definition of a schema object
- [DraftValidation](reference/DraftValidation.html): Draft a validation plan for a given table using an LLM

#### Validation Steps
> Validation steps are sequential validations on the target data. Call Validate's validation methods to build up a validation plan: a collection of steps that provides good validation coverage.


- [Validate.col_vals_gt](reference/Validate.col_vals_gt.html)
- [Validate.col_vals_lt](reference/Validate.col_vals_lt.html)
- [Validate.col_vals_ge](reference/Validate.col_vals_ge.html)
- [Validate.col_vals_le](reference/Validate.col_vals_le.html)
- [Validate.col_vals_eq](reference/Validate.col_vals_eq.html)
- [Validate.col_vals_ne](reference/Validate.col_vals_ne.html)
- [Validate.col_vals_between](reference/Validate.col_vals_between.html)
- [Validate.col_vals_outside](reference/Validate.col_vals_outside.html)
- [Validate.col_vals_in_set](reference/Validate.col_vals_in_set.html)
- [Validate.col_vals_not_in_set](reference/Validate.col_vals_not_in_set.html)
- [Validate.col_vals_increasing](reference/Validate.col_vals_increasing.html)
- [Validate.col_vals_decreasing](reference/Validate.col_vals_decreasing.html)
- [Validate.col_vals_null](reference/Validate.col_vals_null.html)
- [Validate.col_vals_not_null](reference/Validate.col_vals_not_null.html)
- [Validate.col_vals_regex](reference/Validate.col_vals_regex.html)
- [Validate.col_vals_within_spec](reference/Validate.col_vals_within_spec.html)
- [Validate.col_vals_expr](reference/Validate.col_vals_expr.html)
- [Validate.col_exists](reference/Validate.col_exists.html)
- [Validate.col_pct_null](reference/Validate.col_pct_null.html)
- [Validate.rows_distinct](reference/Validate.rows_distinct.html)
- [Validate.rows_complete](reference/Validate.rows_complete.html)
- [Validate.col_schema_match](reference/Validate.col_schema_match.html)
- [Validate.row_count_match](reference/Validate.row_count_match.html)
- [Validate.col_count_match](reference/Validate.col_count_match.html)
- [Validate.data_freshness](reference/Validate.data_freshness.html)
- [Validate.tbl_match](reference/Validate.tbl_match.html)
- [Validate.conjointly](reference/Validate.conjointly.html)
- [Validate.specially](reference/Validate.specially.html)
- [Validate.prompt](reference/Validate.prompt.html)

#### Aggregation Steps
> These validation methods check aggregated column values (sums, averages, standard deviations) against fixed values or column references.


- [Validate.col_sum_gt](reference/Validate.col_sum_gt.html)
- [Validate.col_sum_lt](reference/Validate.col_sum_lt.html)
- [Validate.col_sum_ge](reference/Validate.col_sum_ge.html)
- [Validate.col_sum_le](reference/Validate.col_sum_le.html)
- [Validate.col_sum_eq](reference/Validate.col_sum_eq.html)
- [Validate.col_avg_gt](reference/Validate.col_avg_gt.html)
- [Validate.col_avg_lt](reference/Validate.col_avg_lt.html)
- [Validate.col_avg_ge](reference/Validate.col_avg_ge.html)
- [Validate.col_avg_le](reference/Validate.col_avg_le.html)
- [Validate.col_avg_eq](reference/Validate.col_avg_eq.html)
- [Validate.col_sd_gt](reference/Validate.col_sd_gt.html)
- [Validate.col_sd_lt](reference/Validate.col_sd_lt.html)
- [Validate.col_sd_ge](reference/Validate.col_sd_ge.html)
- [Validate.col_sd_le](reference/Validate.col_sd_le.html)
- [Validate.col_sd_eq](reference/Validate.col_sd_eq.html)

#### Column Selection
> Use the `col()` function along with column selection helpers to flexibly select columns for validation. Combine `col()` with `starts_with()`, `matches()`, etc. for selecting multiple target columns.


- [col](reference/col.html): Helper function for referencing a column in the input table
- [starts_with](reference/starts_with.html): Select columns that start with specified text
- [ends_with](reference/ends_with.html): Select columns that end with specified text
- [contains](reference/contains.html): Select columns that contain specified text
- [matches](reference/matches.html): Select columns that match a specified regular expression pattern
- [everything](reference/everything.html): Select all columns
- [first_n](reference/first_n.html): Select the first `n` columns in the column list
- [last_n](reference/last_n.html): Select the last `n` columns in the column list
- [expr_col](reference/expr_col.html): Create a column expression for use in `conjointly()` validation

#### Segment Groups
> Combine multiple values into a single segment using `seg_*()` helper functions.


- [seg_group](reference/seg_group.html): Group together values for segmentation

#### Interrogation and Reporting
> The validation plan is executed when `interrogate()` is called. After interrogation, view validation reports, extract metrics, or split data based on results.


- [Validate.interrogate](reference/Validate.interrogate.html)
- [Validate.set_tbl](reference/Validate.set_tbl.html)
- [Validate.get_tabular_report](reference/Validate.get_tabular_report.html)
- [Validate.get_step_report](reference/Validate.get_step_report.html)
- [Validate.get_json_report](reference/Validate.get_json_report.html)
- [Validate.get_sundered_data](reference/Validate.get_sundered_data.html)
- [Validate.get_data_extracts](reference/Validate.get_data_extracts.html)
- [Validate.all_passed](reference/Validate.all_passed.html)
- [Validate.assert_passing](reference/Validate.assert_passing.html)
- [Validate.assert_below_threshold](reference/Validate.assert_below_threshold.html)
- [Validate.above_threshold](reference/Validate.above_threshold.html)
- [Validate.n](reference/Validate.n.html)
- [Validate.n_passed](reference/Validate.n_passed.html)
- [Validate.n_failed](reference/Validate.n_failed.html)
- [Validate.f_passed](reference/Validate.f_passed.html)
- [Validate.f_failed](reference/Validate.f_failed.html)
- [Validate.warning](reference/Validate.warning.html)
- [Validate.error](reference/Validate.error.html)
- [Validate.critical](reference/Validate.critical.html)

#### Inspection and Assistance
> Functions for getting to grips with a new data table. Use DataScan for a quick overview, `preview()` for first/last rows, `col_summary_tbl()` for column summaries, and `missing_vals_tbl()` for missing value analysis.


- [DataScan](reference/DataScan.html): Get a summary of a dataset
- [preview](reference/preview.html): Display a table preview that shows some rows from the top, some from the bottom
- [col_summary_tbl](reference/col_summary_tbl.html): Generate a column-level summary table of a dataset
- [missing_vals_tbl](reference/missing_vals_tbl.html): Display a table that shows the missing values in the input table
- [load_dataset](reference/load_dataset.html): Load a dataset hosted in the library as specified table type
- [get_data_path](reference/get_data_path.html): Get the file path to a dataset included with the Pointblank package
- [connect_to_table](reference/connect_to_table.html): Connect to a database table using a connection string
- [print_database_tables](reference/print_database_tables.html): List all tables in a database from a connection string

#### Table Pre-checks
> Helper functions for use with the `active=` parameter of validation methods. These inspect the target table before a step runs and conditionally skip the step when preconditions are not met.


- [has_columns](reference/has_columns.html): Check whether one or more columns exist in a table
- [has_rows](reference/has_rows.html): Check whether a table has a certain number of rows

#### YAML
> Functions for using YAML to orchestrate validation workflows.


- [yaml_interrogate](reference/yaml_interrogate.html): Execute a YAML-based validation workflow
- [validate_yaml](reference/validate_yaml.html): Validate YAML configuration against the expected structure
- [yaml_to_python](reference/yaml_to_python.html): Convert YAML validation configuration to equivalent Python code

#### Utility Functions
> Functions for accessing metadata about the target data and managing configuration.


- [get_column_count](reference/get_column_count.html): Get the number of columns in a table
- [get_row_count](reference/get_row_count.html): Get the number of rows in a table
- [get_action_metadata](reference/get_action_metadata.html): Access step-level metadata when authoring custom actions
- [get_validation_summary](reference/get_validation_summary.html): Access validation summary information when authoring final actions
- [write_file](reference/write_file.html): Write a Validate object to disk as a serialized file
- [read_file](reference/read_file.html): Read a Validate object from disk that was previously saved with `write_file()`
- [ref](reference/ref.html): Reference a column from the reference data for aggregate comparisons

#### Test Data Generation
> Generate synthetic test data based on schema definitions. Use `generate_dataset()` to create data from a Schema object.


- [generate_dataset](reference/generate_dataset.html): Generate synthetic test data from a schema
- [int_field](reference/int_field.html): Create an integer column specification for use in a schema
- [float_field](reference/float_field.html): Create a floating-point column specification for use in a schema
- [string_field](reference/string_field.html): Create a string column specification for use in a schema
- [bool_field](reference/bool_field.html): Create a boolean column specification for use in a schema
- [date_field](reference/date_field.html): Create a date column specification for use in a schema
- [datetime_field](reference/datetime_field.html): Create a datetime column specification for use in a schema
- [time_field](reference/time_field.html): Create a time column specification for use in a schema
- [duration_field](reference/duration_field.html): Create a duration column specification for use in a schema
- [profile_fields](reference/profile_fields.html): Create a dict of string field specifications representing a person profile

#### Prebuilt Actions
> Prebuilt action functions for common notification patterns.


- [send_slack_notification](reference/send_slack_notification.html): Create a Slack notification function using a webhook URL