Changelog

Release	Change type	Description
2024.03.29	Feature	Add hansken.remote.ProjectContext.delete_clean_data to delete cleaned data from the cache, and the reclean param to hansken.remote.ProjectContext.queue_cleaning to delete clean data before cleaning
2024.03.20	Feature	Maintain blocking behaviour of hansken.remote.Connection.delete_project and hansken.remote.Connection.unlink_image for latest Hansken release
2024.02.14	Feature	Add argument engine to hansken.remote.Connection.extract to (optionally) allow the selection of a different extraction engine
2024.02.05	Feature	Expose SearchResult.num_results_quality, TraceletSearchResult.num_results_quality, ValueSearchResult.num_results_quality and FacetResult.count_error
2024.01.11.1	Feature	Add arguments include_image_data and image_id to export to enable portable case
2024.01.11	Feature	Add export/import command to start an export/import
2023.12.15	Documentation	Improve documentation on connection certificate validation
2023.11.28	Feature	Add hansken.remote.Connection.queue_cleaning to put traces on the cleaner queue based on the provided search request
2023.10.31	Feature	Add argument pre_cleaning_priority to hansken.remote.Connection.extract to support pre-clean priority
2023.09.15	Feature	Add argument pre_cleaning to hansken.remote.Connection.extract to toggle pre-cleaning of extracted traces
2023.09.14	Deprecation	Stop adding property user to images in hansken.remote.Connection.create_image
2023.08.08	Feature	Add hansken.remote.Connection.reindex_project to change the number of shards for a project index
2023.06.27	Feature	Enable retrieval of data stream descriptor through hansken.remote.ProjectContext.descriptor and hansken.trace.Trace.descriptor
2023.06.05.1	Bug fix	Use new REST path for regular search requests
2023.06.05	Bug fix	Use importlib.resources rather than pkg_resources to read version, avoiding the runtime deprecation warning
2023.05.01	Bug fix	Prefer hansken.remote.Search.num_results over len(results) for bulk export
2023.03.29	Feature	Add hansken.remote.Connection.add_trace_data to import a data-stream into a single trace
2023.03.06	Documentation	Fix syntax errors in documentation of export recipe functions
2023.02.27	Bug fix	Only send query property in request body when deleting traces
2023.02.07	Feature	Add hansken.remote.Connection.delete_traces to delete traces and their children within a project
2023.02.03	Bug fix	Respect --streaming flag when using CLI hansken upload
2023.01.03	Feature	Support retrieving the export key of a task through hansken.remote.Connection.task_export_key
2022.12.14	Feature	Support deduplication of trace results during search
2022.09.08	Feature	Enable passing a text stream object (e.g. io.StringIO) to hansken.recipes.export.to_csv
2022.08.22	Bug fix	Log a warning when the update method is called more than once per trace
2022.08.15	Documentation	Improve documentation of update and import requests, noting that all changes to an individual trace should be bundled as a single update or import remote call
2022.07.27.1	Bug fix	Work around partially processed chunks during upload, retry only the missing part of the failing chunk
2022.07.27	Bug fix	Avoid logging warnings when iterating over a hansken.remote.SearchResult that had 0 total results (regardless of the number of results being requested and returned by the search call through count=N)
2022.06.16	Bug fix	Auto-retry chunk during upload in case of network errors
2022.06.08	Feature	Add INFO-level log message during hansken.tool.run containing version information
2022.05.30	Bug fix	Tail console output captured from hansken-image-tool during upload, limiting memory usage in cases where hansken-image-tool produces a lot of console output
2022.05.24	Feature	Enable callable user-supplied data stream selector for hansken.recipes.export.bulk to return a falsy value, omitting the trace from the export
2022.03.01.2	Feature	Rely on imagetool defaults, for compatibility with future image tool releases, providing AES/GCM encryption using SHA2-512 stored digests for improved performance
2022.03.01	Bug fix	Wrap a tracelet types / values (like prediction or identity) along the same lines as singular trace types
2022.01.26	Bug fix	Avoid 406 response during authentication; a failed authentication attempt now propagates a 401 (Unauthorized) response error
2022.01.21	Bug fix	Attempt to resume search result stream when a requests.exceptions.ChunkedEncodingError is encountered
2022.01.18	Feature	hansken.remote.SearchResult is also falsy if it is empty when created
2022.01.13	Feature	Remove partially uploaded image when upload is interrupted
2021.12.15	Bug fix	Use builtin float instead of decimal.Decimal while parsing JSON data from the wire
2021.11.29	Bug fix	Enable HTML-only use of hansken.recipes.report when weasyprint is not available (see Installation)
2021.11.19	Feature	Add hansken.remote.Connection.current_user to retrieve information on the currently authenticated user
2021.10.25	Feature	Link image to project before data upload starts, to validate the permissions and reduce the amount of orphaned image data
2021.10.11	Feature	Add the ability to sort search results on similarity to a specified vector, see explanation for details
2021.09.09	Bug fix	Decode streaming chunked and / or gzipped responses for all searching calls
2021.08.31	Feature	Reintroduce chunked upload and set as default, previously removed from the upload command
2021.08.11.1	Feature	Auto-translate vector values to hansken.util.Vector, acting like a sequence of floating point numbers
2021.08.11	Packaging	Move dependencies for the shell command to a new “shell” extra (see Installation)
2021.08.09	Bug fix	Force potential re-authentication before starting upload data stream for the upload command
2021.06.18	Bug fix	Put multiple project ids in the request body instead of the url, avoiding potential excessively long urls
2021.06.17	Feature	Add hansken.query.Trace query to filter tracelet-centered queries by their corresponding traces
2021.06.10	Bug fix	Treat missing shared library for FUSE as an import error, disabling the mount command
2021.05.26	Bug fix	Avoid crashes when updating a trace with data only
2021.05.20.2	Bug fix	Bump minimum Python version in deb configuration
2021.05.20.1	Feature	Add option to upload command to specify image ID with the --image-id argument.
2021.05.20	Feature	Add timeout parameter to search calls
2021.04.13	Feature	Add hansken.query.Regex query type to express regular expression queries
2021.03.31	Bug fix	Avoid buffering of data and uploading in chunks, forward data directly for the upload command
2021.03.15	Bug fix	User backup key should be base64 encoded
2021.03.10	Feature	Support singlefiles
2021.03.09	Bug fix	When checking when the last backup is performed, failed tasks should be ignored.
2021.03.01	Documentation	Fix erroneous statement on included tools for hansken.remote.Connection.extract and miscellaneous other documentation clarifications
2021.02.09	Feature	Add option hansken.query.Facet.include_total to return total number of facet matches, by default disabled. Note that this means that hansken.remote.FacetResult.total is now always None by default. To get the old behaviour back, set hansken.query.Facet.include_total to True
2021.02.08	Bug fix	Avoid raising errors from iterating an exhausted hansken.remote.SearchResult, fixing result.takeone() and result.take(42) called on an exhausted search result
2021.02.04	Bug fix	Backup command with --last-modified should properly match queuedBy name
2021.01.21	Bug fix	Last modification date is ignored when date is more than 56 days back
2021.01.20	Bug fix	Backup file is not downloaded when keys-folder is specified
2021.01.14.1	Bug fix	Backup process should continue with other projects after error
2021.01.14	Feature	Add partial support for multi project search
2021.01.13	Bug fix	Avoid crashes on unparseable dates, now logs a warning and falls back to datetime.min in UTC
2021.01.12	License	Distribute hansken.py under the Apache License 2.0
2021.01.11	Bug fix	Backup command should handle keystore argument, create download folder
2020.12.14.1	Bug fix	Wrap a ‘chunked’ search response before handing it to the JSON parser
2020.12.14	Bug fix	Update backup api call according to endpoint definition
2020.11.13	Documentation	Include hansken.remote.ValueSearchResult in the documentation (return type for hansken.remote.ProjectContext.unique_values)
2020.10.27	Bug fix	Make hansken.trace.Privileged enum values str, enabling my_trace.privileged = 'suspected'
2020.09.29	Feature	Add hansken.trace.TraceBuilder.child_builder to aid creating new traces descending from other new traces
2020.08.06	Feature	Add backup command to start a backup”
2020.07.31	Feature	Add a debug mode to hansken.trace.TraceBuilder, logging trace to be created instead of actually creating it
2020.07.29	Deprecation	Deprecate hansken.remote.SearchResult’s builtin length (len), mark result as falsy when / after it’s been exhausted (use new num_results property)
2020.07.23	Bug fix	Use preference endpoint supplied on the command line in hansken.tool.run and applicable sub commands
2020.07.16	Feature	Enable exclusion of properties for project clone requests to hansken.remote.Connection.clone_project
2020.07.09	Feature	Delete project specific preferences on delete project
2020.06.22.1	Bug fix	Make (path to) the hansken-image-tool customizable for usage in Windows
2020.06.22	Feature	Auto-translate latitude/longitude values to hansken.util.GeographicLocation, allowing easier access to the separate latitude / longitude fields Note that this might break scripts that rely on latitude/longitude values to be offered as a str. The new hansken.util.GeographicLocation will coerce to the same str-value used before.
2020.06.03.1	Feature	Enable managing Hansken resources through hansken.remote.Connection.create_resource and friends
2020.04.06	Bug fix	Set dependency on ijson to <3, fixing compatibility issue (ImportError mentioning items)
2020.03.16	Documentation	Improve documentation on hansken.remote.ProjectContext.unique_values, clarify selecting and querying from tracelet properties
2020.03.03	Bug fix	Retain auto-resume on use of parameter after in hansken.remote.ProjectContext.unique_values
2020.02.13	Documentation	Clarify documentation on logging exceptions and the on_error argument to hansken.recipes.export.bulk
2020.02.07	Feature	Enable determining stream to be exported per trace by supplying stream as a callable to hansken.recipes.export.bulk
2020.01.15	Feature	Log warnings retrieved from a search result as warnings from hansken.py
2020.01.13	Bug fix	Auto-resume search result when broken by ijson.backends.python.UnexpectedSymbol errors
2019.12.16	Feature	Enable deletion of images linked to a project when deleting a project through hansken.remote.Connection.delete_project
2019.11.21	Feature	Improve usability of snippets by automatically adding keys to snippet request objects and specifying highlight terms to multiple requests at a time for hansken.remove.ProjectContext.snippets
2019.11.04	Feature	Enable retrieval of unique values for a property (also known as a value search) through hansken.remote.ProjectContext.unique_values
2019.10.29	Bug fix	Accept a 404 response when logging out of a service that does not support session management
2019.10.02	Bug fix	Encode tags in url so it is possible to set tags containing a hash symbol
2019.08.19	Bug fix	Pass colors='neutral' to embedded ipython shell for the shell command to restore colors to the interactive shell
2019.08.09	Feature	Add optional parameters message_type and task_id to hansken.remote.Connection.log_messages, enable retrieval of failed traces or restricting messages to a specific task
2019.07.30	Bug fix	Fix broken auto-resume for tracelet searches, pass the requested tracelet_type to the result iterator
2019.07.19	Bug fix	Accept new style response (201, created) of clone REST call
2019.07.17.1	Bug fix	Serialize hansken.query.Tracelet query type to the right wire format
2019.07.17	Feature	Enable searching for tracelets (e.g. entities) by adding hansken.remote.ProjectContext.search_tracelets
2019.07.11	Feature	Add command line options to control buffer size, number of retries and wait time for the upload command
2019.07.02	Bug fix	Match user identifiers in mixed-auth environments (e.g. non-authenticated keystore) better, avoid checking against anonymous user
2019.07.01	Feature	Enable adding data streams to a hansken.trace.TraceBuilder with hansken.trace.TraceBuilder.add_data
2019.06.17	Bug fix	Use new REST format for the tasks command, incorporating the position of a task on the queue in the output
2019.06.01	Deprecation	Drop support for Python 2.7
2019.05.13	Bug fix	Force authentication on (automatic) calls to hansken.remote.Connection.open to avoid simultaneous authentication attempts in a multi-threaded environment
2019.05.06.1	Documentation	Add examples and external links to documentation on report recipe
2019.05.06	Bug fix	Fix minor spelling and capitalization errors in the quickstart command
2019.04.02	Feature	Add hansken.py calls for adding, removing, retrieving and updating preferences Note that the preference service is a separate REST endpoint, which needs to be configured. This can be done by setting the preference argument.
2019.02.26	Deprecation	Issue PendingDeprecationWarning on Python 2.7, support for Python 2.7 will be dropped for version 2019.06.01
2019.02.25	Feature	Add hansken.query.Tracelet query type to express queries to match within the same tracelet instance
2019.02.13	Bug fix	Include template files with the hansken.recipes.report module (distribution packages were shipped without templates, breaking the module entirely)
2019.02.07.2	Feature	Add --include-tools and --exclude-tools command line options to the extract command
2019.02.07.1	Feature	Add --skip-images option to extract command to skip particular images
2019.02.07	Bug fix	Restrict number of or-clauses in query for user-generated content in projects with many images for the extract command
2019.01.25	Deprecation	Deprecate hansken.admin module, all functionality has been moved to hansken.remote.Connection (note that administrative tasks generally require additional rights)
2019.01.07	Feature	Add hansken.connect.connect and hansken.connect.connect_project as simple alternative to hansken.tool.run
2019.01.04	Feature	Use chunked upload for the upload command to improve performance when uploading data through the Hansken REST API
2018.12.19	Bug fix	Apply modeled data type conversion for mapped type properties when using hansken.trace.Trace.get (e.g. trace.get('tool.meta.publishedOn'), which should be a datetime object)
2018.12.13	Feature	Enable use of filtered sort for searches
2018.12.10	Feature	Make hansken.py aware of cardinality definition changes to the Hansken data model
2018.12.06.1	Bug fix	Read hansken.trace.Trace.creator from new-style metadata (‘toolrun’ type), falling back to old-style metadata (‘tool’ type), report unknown version instead of None for creator when version is unavailable
2018.12.06	Bug fix	Read hansken.trace.Trace.tool_versions from new-style metadata (‘toolrun’ type), falling back to old-style metadata (‘tool’ type)
2018.11.30.1	Feature	Parallellize hansken.recipes.export.bulk internally to speed up large scale data exports Note that this is enabled by default, requiring that the callable to do the actual retrieval and writing of data is thread-safe. Deterministic ordering of traces being exported is also effected. To turn parallellism off, pass jobs=False to hansken.recipes.export.bulk.
2018.11.30	Bug fix	Avoid crashing the quickstart command when a Keycloak IDP is selected
2018.11.22	Bug fix	Make hansken.trace.TraceTypeView objects behave more like the Mapping they’re supposed to be, fixes incorrect len(trace.file) and broken trace.file.items()
2018.11.16	Bug fix	Default hansken.recipes.export.to_csv to UTF-8 encoded output, avoid using platform encoding on Windows
2018.11.12	Bug fix	Avoid consuming search results when key "facets" is missing from the search result stream, default to no facets ([])
2018.11.09	Bug fix	Allow specifying a fallback text encoding for hansken.recipes.export.data_stream exports (defaults to warning about omitting data when encoding cannot be determined)
2018.11.08	Bug fix	Fix hansken.trace.TraceBuilder.update reporting unknown trace property for data-type mapped types (e.g. data.raw.hash.sha256)
2018.11.07	Feature	Add hansken.trace.ProjectContext.suggest to let Hansken expand search terms / queries from the index
2018.11.05	Feature	Add hansken.trace.TraceBuilder to more easily create and update trace metadata and utility functions to create builders from trace and project point of views
2018.11.01	Bug fix	Fix deprecated imports of abstract collection base classes (would break on Python 3.8+)
2018.10.30	Bug fix	Remove use of weak references in hansken.auth, prevent the authentication handler from losing access to its session object
2018.10.29	Bug fix	Avoid auto-closing a SearchResult on iteration, auto-close when stream is exhausted (also improve error message when re-iterating an exhausted SearchResult)
2018.10.25	Deprecation	Rename run_x methods in command line modules to run_x_command (old versions available, will generate deprecation warnings)
2018.10.08	Feature	Add hansken.trace.TraceModel wrapper to help with handling and expanding trace model property definitions
2018.10.05	Deprecation	Drop development support for Python 3.4 (runtime should be unaffected)
2018.08.29	Feature	Add tasks command to command line to list tasks tracked by the Hansken scheduler
2018.08.27	Feature	Add hansken.py calls exposing scheduled tasks
2018.08.23	Bug fix	Accept new style responses of extraction REST calls
2018.07.31	Bug fix	Include ijson.common.IncompleteJSONError in list of errors triggering auto-resume of hansken.remote.SearchResult
2018.07.30	Feature	Add hansken.remote.Connection.create_trace to add a new child trace to an existing parent
2018.07.23	Bug fix	Show exception traceback by default for calls to hansken.tool.run, builtin commands still hide the traceback by default
2018.06.15	Feature	Enable requesting results from ‘broken’ indices (this would trigger an error by default, override with argument incomplete_results=True for search calls)
2018.05.17	Feature	Add shorthands for creating PDF documents to the reporting recipe
2018.04.26	Bug fix	Align type conversions for attribute and hansken.trace.Trace.get() access, converting from wire format in both cases
2018.04.20	Feature	Add a reporting recipe as module hansken.recipes.report
2018.04.11	Bug fix	Fix resolving data size in hansken.trace.Trace.snippets for modeled traces
2018.03.07	Feature	Enable importing data for existing traces Note that data types need be named; arguments for hansken.remote.Connection.import_trace and hansken.remote.ProjectContext.import_trace are dict s. Additionally, only data type names defined in the data model (names like raw or plain) will be accepted by the server-side.
2018.03.01	Documentation	Let the --no-verify command line option to disable certificate verification show up in --help output
2018.02.20	Feature	Enable overriding 3rd party HTTP/HTTPS connection pool size as --connection-pool-size command line argument or connection_pool_size keyword argument to hansken.tool.run or hansken.remote classes This is only useful when running many parallel requests using hansken.py, causing “Connection pool is full, discarding connection” warnings. Note that these warnings are are not harmful to the script being run, but increasing the connection pool size gets rid of these messages and optimizes resource use for both hansken.py and Hansken’s server side.
2018.02.14	Bug fix	Fix unhashable type: 'DictView' in hansken versions command and hansken.trace.Trace.image_id property
2018.02.06	Bug fix	Enable passing logging timezone as a string to hansken.tool.run, e.g. run(..., timezone='Europe/Amsterdam')
2018.01.17	Deprecation	Rename hansken.tool.create_parser hansken.tool.create_argument_parser, add deprecation warning on old function
2018.01.12	Feature	Add properties parent_id, parent_uid and parent to hansken.trace.Trace Note that hansken.trace.Trace.parent makes a call to remote to retrieve the full metadata of a trace’ parent. This replaces the old behaviour where the parent property of a trace would contain the trace’ parent uid.
2018.01.09	Feature	Include a changelog in the generated documentation
2017.12.14	Bug fix	Fix authentication realm detection by allowing OpenAM identity provider URLs to end in variations of /idp
2017.11.30	Bug fix	Fix import errors introduced by ‘robust search’ workaround
2017.11.23	Bug fix	Make sure hansken.recipes.export.safe_name uses project-wide unique names
2017.11.22	Feature	Add hansken.recipes.export.data_stream to export trace data alongside metadata for CSV exports
2017.11.15	Bug fix	Fix infinite loop in hansken.remote.SearchResult when using a result offset or cutoff
2017.11.10.1	Bug fix	Pass falsy values (like min=0 in a hansken.query.RangeFacet) through REST calls
2017.11.10	Feature	Use authenticated upload through Hansken REST API in the command line upload command
2017.11.08	Workaround	Integrate the ‘robust search’ workaround into hansken.remote.SearchResult, automatically attempt resuming the stream from the last successfully processed result
2017.10.18	Feature	Enable uploading data through the REST interface Protocol level violations caused by server-side errors are automatically detected while reading a search result stream. Auto-resume is enabled when a search call is made without an offset (start keyword) or cutoff (count keyword).
2017.10.10	Feature	Add hansken.trace.Trace.update shorthand calling trace import REST API call to ‘import’ a single property or a number of properties Note that the trace import REST API call requires permissions typically not provided to regular end users.
2017.09.27	Feature	Enable username + password authentication for Keycloak identity providers
2017.09.25	Bug fix	Swap image metadata creation and starting data upload to support new server-side requirements
2017.09.15	Bug fix	Use correct REST API path for project clone call
2017.08.17	Bug fix	Unbreak traces with tags in their metadata
2017.08.15	Feature	Support new trace metadata format (user-facing code unchanged, hansken.trace.Trace reacts in the same way)
2017.07.19	Feature	Enable calling project clone REST API call from hansken.admin.Admin
2017.06.15.3	Bug fix	Use “username” consistently, renaming some “user”-like command line arguments
2017.06.15.2	Bug fix	Interpret count=None as ‘no cutoff’ for search calls, resulting in all available results
2017.06.15.1	Feature	Add hansken.trace.Trace.preview shortcut to select preview data by mime type
2017.06.13	Feature	Enable selection of metadata properties to be returned in a search result
2017.06.08	Feature	Enable retrieval of text / data snippets through hansken.remote.ProjectContext.snippets and hansken.trace.Trace.snippets
2017.05.29	Feature	Support geohashing facets as hansken.query.GeohashFacet
2017.05.15	Feature	Support image conversion status reported by hansken-image-tool This reduces the amount of data to be sent from the server-side and the amount of data to be parsed by hansken.py. All metadata types of a trace are available on the corresponding hansken.trace.Trace object, but all properties not selected in the search call will be None.
2017.05.10	Feature	Expose a hansken.trace.Trace’ available data types as trace.data_types
2017.04.10	Feature	Cache retrieved data keys on the hansken.remote.ProjectContext object
2017.03.09	Feature	Make hansken.remote.ProjectContext auto-deactivate a project if it was the one to auto-activate it
2017.02.17	Feature	Enable calling cluster health REST call from hansken.admin.Admin
2017.02.16.1	Feature	Enable export partial data in hansken.recipes.export.to_file, using keywords offset and size
2017.02.10	Feature	Enable starting a partial extraction from hansken.admin.Admin.extract
2017.02.09	Feature	Enable use of custom command line argument parser combined with project context callback in hansken.tool.run To ensure working with the rest of the machinery inside hansken.tool.run, use hansken.tool.create_parser to bootstrap the command line parser. After adding any command line arguments to that parser, the user-provided callback (like with the with_context keywork argument) is called with two keyword arguments: context and args, see hansken.tool.run.
2017.02.08	Feature	Enable logging in timezone other than UTC with -z/--timezone command line flag
2017.01.26	Feature	Automatically request a project index to be opened when REST API reports it being closed
2017.01.18	Feature	Remove 5s timer after certificate validation being disabled
2017.01.03.1	Bug fix	Define command line mount command regardless of fusepy being installed, produce error message when used without required dependency
2017.01.03	Bug fix	Encode password prompt to platform default encoding on Python 2 in Windows