Skip to content
Back to projects

University2025

Backup Application

Automated client-server backup system with version management, AES-256-GCM encryption and Flask web interface

Duration
3 months
Team
University project, team of 4 students
My role
Lead Developer, System architecture, version management, encryption, web interface
TechnologiesPythonSSH/ParamikoSCPFlaskSQLiteCryptography (AES-256-GCM)GzipWatchdog
Backup Application

Context

Computer Science degree project aimed at creating a robust backup system to protect data against ransomware and user errors, with 30-day retention.

Objectives

Develop an automated backup system, secured via SSH, with intelligent version management, gzip compression (70% reduction), SHA256 hash deduplication and AES-256-GCM encryption.

Technical approach

Client-server architecture with client-side monitoring daemon (watchdog), secure SSH/SCP transfer, server-side processing (compression + encryption + deduplication), SQLite metadata storage and Flask web interface for monitoring and restoration.

Architecture

Client: watchdog daemon, SSH/SCP via Paramiko. Server: process_file.py for processing, version_manager.py for version/deduplication management, encryption.py for AES-256-GCM, Flask app.py for web interface (dashboard, restore, REST API), SQLite for metadata, systemd service for automation.

Skills developed

Advanced Python & System Architecture

Client-server architecture with watchdog daemon, SSH/SCP transfers via Paramiko, and systemd automation.

Cryptography & Security

AES-256-GCM encryption with PBKDF2 key derivation, gzip compression (70% reduction), and SHA256 hash deduplication.

Flask & REST API

Flask web interface with real-time dashboard, REST API for version browsing and restoration, integrated authentication.

SQLite Database

Version metadata storage with timestamps, 30-day retention management, and optimized queries.

Code highlights

Version management with compression and encryption
python
1# daemon-sauvegarde - Version Manager avec deduplication
2class VersionManager:
3    def save_version(self, file_path, relative_path, action='modified'):
4        # 1. Calcul hash SHA256 du fichier
5        file_hash = self._calculate_hash(file_path)
6        file_size = os.path.getsize(file_path)
7
8        # 2. Verification changement (compare avec version actuelle)
9        current_hash = self._get_current_hash(relative_path)
10        if current_hash == file_hash:
11            return  # Fichier inchange, pas de nouvelle version
12
13        # 3. Deduplication - verifier si hash existe deja
14        dedup_info = self._check_deduplication(file_hash)
15
16        if dedup_info:
17            # Reutiliser fichier existant (incrementer ref_count)
18            self._increment_ref_count(file_hash)
19            compressed_size = dedup_info['compressed_size']
20            dedup_ref = dedup_info['dedup_path']
21        else:
22            # 4. Compression gzip (niveau 6, ~70% reduction)
23            temp_compressed = f"{file_path}.gz"
24            with open(file_path, 'rb') as f_in:
25                with gzip.open(temp_compressed, 'wb', compresslevel=6) as f_out:
26                    shutil.copyfileobj(f_in, f_out)
27
28            # 5. Chiffrement AES-256-GCM
29            encrypted_path = f"{temp_compressed}.enc"
30            self.encryption_manager.encrypt_file(temp_compressed, encrypted_path)
31
32            # 6. Stockage deduplication (hash-based path)
33            dedup_ref = self._store_deduplicated(file_hash, encrypted_path)
34            compressed_size = os.path.getsize(encrypted_path)
35
36        # 7. Enregistrement version dans SQLite
37        timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S-%f')
38        self.cursor.execute("""
39            INSERT INTO file_versions
40            (file_path, version_timestamp, file_size, compressed_size,
41             file_hash, dedup_ref, is_compressed, is_encrypted, action)
42            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
43        """, (relative_path, timestamp, file_size, compressed_size,
44              file_hash, dedup_ref, True, True, action))
45
46        self.conn.commit()

This code implements version management with intelligent deduplication. Each file is hashed (SHA256), compared to current version, then compressed (gzip 70%), encrypted (AES-256-GCM) and stored in deduplicated manner. If same hash already exists, file is reused (space saving). Metadata is recorded in SQLite with timestamp for 30-day retention.

Flask API for version restoration
python
1# daemon-sauvegarde - Flask Web Interface API
2@app.route('/api/files/<path:file_path>/versions', methods=['GET'])
3@auth.login_required
4def get_file_versions(file_path):
5    """Recupere toutes les versions d'un fichier"""
6    versions = vm.cursor.execute("""
7        SELECT version_timestamp, file_size, compressed_size,
8               file_hash, action, created_at
9        FROM file_versions
10        WHERE file_path = ?
11        ORDER BY version_timestamp DESC
12    """, (file_path,)).fetchall()
13
14    return jsonify({
15        'file_path': file_path,
16        'versions': [{
17            'timestamp': v[0],
18            'size': v[1],
19            'compressed_size': v[2],
20            'hash': v[3],
21            'action': v[4],
22            'created_at': v[5]
23        } for v in versions]
24    })
25
26@app.route('/api/restore', methods=['POST'])
27@auth.login_required
28def restore_file():
29    """Restaure une version specifique d'un fichier"""
30    data = request.json
31    file_path = data.get('file_path')
32    timestamp = data.get('timestamp')
33
34    # 1. Recuperer infos version depuis SQLite
35    version = vm.cursor.execute("""
36        SELECT dedup_ref, is_compressed, is_encrypted
37        FROM file_versions
38        WHERE file_path = ? AND version_timestamp = ?
39    """, (file_path, timestamp)).fetchone()
40
41    if not version:
42        return jsonify({'error': 'Version not found'}), 404
43
44    dedup_ref, is_compressed, is_encrypted = version
45
46    # 2. Charger fichier depuis dedup_store
47    stored_file = os.path.join(vm.backup_path, dedup_ref)
48
49    # 3. Dechiffrement AES-256-GCM
50    decrypted_file = f"{stored_file}.dec"
51    vm.encryption_manager.decrypt_file(stored_file, decrypted_file)
52
53    # 4. Decompression gzip
54    restored_file = os.path.join('./restored', file_path)
55    os.makedirs(os.path.dirname(restored_file), exist_ok=True)
56    with gzip.open(decrypted_file, 'rb') as f_in:
57        with open(restored_file, 'wb') as f_out:
58            shutil.copyfileobj(f_in, f_out)
59
60    # 5. Nettoyage temporaires
61    os.remove(decrypted_file)
62
63    return jsonify({
64        'success': True,
65        'restored_path': restored_file,
66        'download_url': f'/api/download/{file_path}'
67    })

This Flask API exposes REST endpoints to consult version history and restore files. GET /api/files/<path>/versions returns all versions with metadata (timestamp, sizes, hash). POST /api/restore handles complete restoration: retrieval from dedup_store, AES-256-GCM decryption, gzip decompression and writing of restored file. The web interface uses these endpoints to enable point-in-time restoration.

Results & outcomes

Production-ready system with 80-90% disk space reduction (70% gzip compression + deduplication), functional web interface with real-time dashboard, complete REST API, automated integrity and restore tests, 30-day retention.

Reflection & learnings

Enriching project that allowed me to deepen my skills in system architecture, security (SSH, encryption), optimization (compression + deduplication) and web development. Version management with retention particularly trained me in data storage and recovery challenges.

Links

Next projectAutoInit and Streamline for Invoices customization at ITESOFT