Project

General

Profile

Actions

Feature #92

open

Feature #82: Implement a work copy of our website

Task #9: Add a cron script to make a static copy of Spip

Define a way to automatically put online a snapshot

Added by Jean-Michel Philippe about 13 years ago. Updated over 12 years ago.

Status:
Ready for test
Priority:
High
Category:
Programming
Target version:
-
Start date:
06/01/2011
Due date:
12/26/2011 (over 12 years late)
% Done:

90%

Estimated time:
2:00 h
Spent time:

Description

Once snapshots are daily generated and purged, we need to analyze them to be able to decide whether a snapshot is correct or not and can go online. The objective is to do it automatically in the future. For this we can look at:

  • the overall snapshot size, of about 15 MB
  • the total number of files in the snapshot

Both should not decrease too much from one snapshot to another. Maybe we should record the full history of these parameters in a log file to be sure we can change algorithm parameters without risking to miss a case already seen.


Related issues 1 (1 open0 closed)

Follows (1 day) Website - Task #90: Purge old website snapshot directoriesReady for testJean-Michel Philippe05/09/201105/16/2011

Actions
Actions #1

Updated by Jean-Michel Philippe about 13 years ago

  • Due date changed from 07/26/2011 to 07/01/2011
  • Status changed from New to In Progress
  • % Done changed from 0 to 20

I added code lines to log some measurements on mirrored website: size and number of files. These figures are logged in to logs/size.log and logs/count.log. There is a rotation process to keep at most 30 entries in log (can be tuned). These values will then be used to compute a statistical decision.

Actions #2

Updated by Jean-Michel Philippe over 12 years ago

  • Due date changed from 07/01/2011 to 12/26/2011
  • % Done changed from 20 to 40

The log rotation process is now ok and no data is lost. I propose the following algorithm for automatic updates of the static website:

  1. compute stats and log them
  2. check that stats are still increasing or stable
  3. check that no previous stats have shown a decrease
  4. put the static copy online
  5. remove snapshots older than 1 month

If the two tests are not all passed, I propose to record the stats in the log files but skip log rotation in order to preserve the full history from the time a decrease was detected. Of course the last two operation are not performed if tests do not pass.

Actions #3

Updated by Jean-Michel Philippe over 12 years ago

  • Status changed from In Progress to Ready for test
  • % Done changed from 40 to 90
  • Estimated time changed from 3:00 h to 2:00 h

The code is online with “dangerous” actions commented out (link replacement and directory removal). Now need to check its functioning during several weeks.

Actions

Also available in: Atom PDF