Purge Policies

the purge policies module allows you to create instance/company purge policies to archive or purge archive data or userdata automatically given a certain condition the purge policies module includes the following sections instance purge settings allows you to add archive or userdata purge policies for the instance level you can set up a policy action to purge or archive data and limit the purge by the number of periods, by the project's data size, or by the free disk space percentage companies purge settings allows you to add archive or userdata purge policies for a company you can set up a policy action to purge or archive data and limit the purge by the number of periods, by the project's data size, or by the free disk space percentage you can also filter the instance/company purge policies list from this sub module archiving needs additional disk space when data is archived, the data is extracted from a clickhouse data server and stored as a csv gz file each csv gz file is up to ten times the size of the data size displayed in litmus edge manager (the data size that is shown in the stats by periods section of docid\ suvgdkwr 7wy9lrvklbqm is a compressed size) before you archive, ensure the litmus edge manager has enough disk space to store all the csv gz file data that it will archive insufficient disk space will prevent the data from being archived after archiving, review the list of archive files to ensure the archive operation was successful access purge policies ui to access the purge policies ui log in to the litmus edge manager (lem) admin console at the following url https //\[lem ip address] 8446 from the navigation panel, select data lifecycle data lifecycle's statistics pane by default appears from data lifecycle's navigation sub panel, select purge policies data lifecycle's purge policies pane appears refer to the following actions you can take on the litmus edge manager admin console data lifecycle's purge policies pane action details \[1] show filter / hide filter search by data type and policy action \[2] create instance purge policy option availability depends on previous selections see the create a policy section below \[3] action menu edit option availability depends on previous selections remove \[4] show filter / hide filter search by company name, data type or policy action \[5] create company purge policy option availability depends on previous selections \[6] action menu edit option availability depends on previous selections remove period explained litmus edge manager keeps track of disk usage in partitions of months the tracked disk usage is only for clickhouse data (hence all archive/purge operations are applicable for only clickhouse data) the disk usage amount is calculated either from the instance or each individual project each month of disk usage is defined as a period for example, your litmus edge manager has five projects to calculate disk usage, each project will be categorized as one period for the same month so litmus edge manager will generate five periods for the same month, each period including one project as an additional example, three months have passed for the five projects above, each of them experienced some disk usage in those three months so you will have 15 additional periods (five projects of disk usage 3 months) periods are also categorized as previous or current if the current system date is march 2023, then all disk usage by projects during march 2023 are current periods all periods from february 2023 and before are previous periods create a policy a policy is only applicable for a previous p eriod see docid\ suvgdkwr 7wy9lrvklbqm note when policy types are considered, disk usage for a current period of the litmus edge manager will be ignored select company select a company to apply to policy towards all of the company's projects and the projects' respective periods will be included when creating a policy this option is only available for companies purge settings select data object archive apply the policy toward an existing archive in docid\ dsjzj4n3ksgiqliabm gy userdata apply the policy towards existing data note for instance purge settings, you may only create one policy per data object for companies purge settings, you may only create one policy per company data object pairing policy action archive archive the data if a previous period fails to stay within a policy type limit you cannot archive an archive data object when this policy is selected, a rchiving a period will also automatically purge the data after successful archiving purge delete the data if a previous period fails to stay within a policy type limit policy type (see the more on policy types section below) limit by number of periods you cannot select this for an archive data data object limit by project's data size free disk space limit, % policy limits value limit by number of periods at least 1 or more limit by project's data size at least 1 or more free disk space limit, % as of litmus edge manager 2 10 1, enter a percentage value between 0 50% decimal values are allowed more on policy types refer to the period explained section above to learn more about the definition of period in litmus edge manager's data lifecycle feature when a policy is selected for instance purge settings, it will check all existing project's previous periods when a policy is selected for company purge settings, it will check the selected company's existing project's previous periods limit by number of periods data will be archived/purged based on the number of months you enter example 1 you enter 1 and the current date is january 1, 2023 litmus edge manager checks the age (in months) of every single previous period so december 2022 would be 1, november 2022 would be 2, and so on it compares the period's age against the age you entered earlier (1) if the period's age is equal to or greater than the age you entered earlier (1), then archive/purge all data within each previous individual month will be archived/purged starting backwards from december 2022 when the current date advances to february 1, 2023, the data from january 2023 will be archived/purged example 2 you enter 3 and the current date is january 1, 2023 litmus edge manager checks the age (in months) of every single previous period so december 2022 would be 1, november 2022 would be 2, and so on it compares the period's age against the age you entered earlier (3) if the period's age is equal to or greater than the age you entered earlier (3), then archive/purge all data within each previous individual month will be archived/purged starting backwards from october 2022 when the current date advances to february 2023, the data from november 2022 will be archived/purged this means the three month duration of december 2022 february 2023 is stored in litmus edge manager limit by project's data size data will be archived/purged for a previous period if the data size is equal to or greater than the data size you enter the o ldest data is archived/purged first example 1 you enter 1gb and the current date is january 1, 2023 litmus edge manager checks the data size of every single previous period if the single previous period's data size is 1gb or more, it will be archived/purged when the current date advances to february 1, 2023, litmus edge manager checks for all the data size for the january 2023 period all january 2023 periods with a data size of 1gb or more are archived/purged free disk space limit, % data will be archived/purged once as long as the free disk space percentage is below the specified threshold the o ldest data is archived/purged first example 1 you enter 1% and the total disk space size is 100gb once litmus edge manager's free disk space is less than 1gb, it will archive/purge all previous periods if litmus edge manager's free disk space is 1gb or more, do nothing for purging, all the purge operations of each period will happen concurrently therefore, if a purge increases the free disk space percentage above the 1gb threshold, it will not stop the purge for the remaining periods example 2 you enter 50% and the total disk space size is 100gb once litmus edge manager's free disk space is less than 50gb, it will archive/purge all previous periods if litmus edge manager's free disk space is 50gb or more, do nothing for purging, all the purge operations of each period will happen concurrently therefore, if a purge increases the free disk space percentage above the 50gb threshold, it will not stop the purge for the remaining periods