added all changes to tests for final submission of fyp, keeping things up to date

Author: kataokaa

test/pyhttpd/ech/README.md

@@ -1,24 +1,98 @@
-### Step 1: Local DNS Configuration
-Map the test domain to your local loopback interface:
-```bash
-echo "127.0.0.1 ech-test.fyp.local" | sudo tee -a /etc/hosts
+# ECH Apache Implementation & Verification Suite
+
+This repository contains the source code, build instructions, and an automated testing suite for Encrypted Client Hello (ECH) within the Apache `httpd` (mod_ssl) directory.
+
+---
+
+## 1. Build & Compilation Guide
+To reproduce the experimental environment, you must build both the OpenSSL fork and Apache from source to ensure the ECH state machine is correctly linked.
+
+### A. Build ECH-Enabled OpenSSL
+We utilize a specific fork of OpenSSL that includes HPKE and ECH protocol support.
+
+# Clone and enter the experimental repository
+git clone (https://github.com/sftcd/openssl.git) openssl-ech
+cd openssl-ech
+
+# Configure for a local prefix to avoid system conflicts
+./config --prefix=$HOME/openssl-ech --openssldir=$HOME/openssl-ech -Wl,-rpath,'$(LIBRPATH)'
+
+# Compile and install
+make -j$(nproc)
+make install
+
+
+B. Build ECH-Enabled Apache (httpd)
+Apache must be linked against the custom OpenSSL build created above.
+git clone (https://github.com/apache/httpd.git) httpd-ech
+cd httpd-ech
+
+# Configure with custom SSL path and static dependency hooks
+./buildconf
+./configure --prefix=$HOME/apache-ech \
+            --enable-ssl \
+            --enable-so \
+            --with-ssl=$HOME/openssl-ech \
+            --enable-mods-shared=all \
+            --enable-ssl-staticlib-deps
+
+make -j$(nproc)
+make install
 
-### Step 2: Environment Initialization
-Set the path to your ECH-enabled OpenSSL build and run the setup script:
-export OPENSSL_ECH_PATH=/path/to/your/openssl
+2. Verification Suite Setup
+A. Shell Environment
+Set these variables in your active terminal to point the verification tools to your custom binaries.
 
-### for me right now
-export OPENSSL_ECH_PATH=/home/yag/final_year/build/openssl
+export OPENSSL_ECH_PATH=$HOME/openssl-ech
+export OPENSSL_CONF=/etc/ssl/openssl.cnf
+export PATH=$OPENSSL_ECH_PATH/bin:$PATH
 export LD_LIBRARY_PATH=$OPENSSL_ECH_PATH/lib64:$LD_LIBRARY_PATH
 
+B. Network Configuration
+echo "127.0.0.1 ech-test.fyp.local" | sudo tee -a /etc/hosts
+
+C. Python Dependencies
+pip install -r requirements.txt
 
+D. Cryptographic Initialization
+Generate the PKI hierarchy (Root CA, Server Certs) and the ECH key material.
+cd scripts/
 ./setup_test_env.sh
+cd ..
 
-### Step 3: Start the Server
+ 3. Infrastructure Orchestration
+Deploy the containerized server. We use a volume-wipe strategy to ensure configuration idempotency.
+cd infrastructure/
+docker-compose down -v
 docker-compose up -d --build
 
-### Step 4: execute test
-pytest -s test_ech.py
+# Initialize a clean configuration backup for robustness testing
+docker exec ech-server cp /usr/local/apache2/conf/httpd.conf /usr/local/apache2/conf/httpd.conf.bak
+cd ..
+
+Running the Full Suite
+pytest -v cases/
+
+Security Audit (Wire-Level)
+To verify that no SNI information leaks in cleartext, run the Tshark-based auditor:
+sudo ./scripts/verify_ech.sh
+
+Project Structure
+/cases: Modular pytest logic.
+
+/infrastructure: Dockerfiles and Apache httpd.conf templates.
+
+/lib: Environment abstractions and Selenium/WebDriver drivers.
+
+/scripts: Setup and wire-level verification tools.
+
+/conf: Storage for generated .pem keys and ECHConfigs.
+
+Success Criteria
+Verification is successful if:
+
+test_01 and test_02 return PASSED (Protocol and Browser Success).
 
+test_04 identifies a Syntax Error (Robustness Success).
 
-Test will succeed if firefox parses the provided ECHConfig, Apache uses the SSLECHKeyDir to decrypt ClientHello, and then routes the decrypted request to the ech-test.fyp.local VirtualHost, avoiding falling back to the public default.
+verify_ech.sh detects zero occurrences of the string ech-test.fyp.local in the cleartext portion of the TLS ClientHello.

test/pyhttpd/ech/cases/__init__.py

@@ -0,0 +1,69 @@
+import pytest
+import subprocess
+from lib.env import EchTestEnv
+
+class TestEchProtocol:
+
+    @pytest.fixture(autouse=True)
+    def setup_method(self):
+        self.env = EchTestEnv()
+        self.config = self.env.get_ech_config()
+
+    # --- SECTION 1: FUNCTIONAL & REGRESSION ---
+
+    def test_01_standard_tls_regression(self):
+        """Verify standard TLS 1.3 works without ECH."""
+        output = self.env.run_openssl(["-brief"])
+        assert "Verification: OK" in output or "return code: 0" in output
+
+    def test_02_protocol_correctness(self):
+        """Verify the server actually accepts a valid ECH extension."""
+        output = self.env.run_openssl(["-brief", "-ech_config_list", self.config])
+        success_markers = ["ECH: accepted", "02 79", "ech required"]
+        assert any(marker in output for marker in success_markers)
+
+    def test_03_protocol_hrr_handling(self):
+        """Verify ECH survives a HelloRetryRequest (HRR)."""
+        output = self.env.run_openssl(["-ech_config_list", self.config, "-groups", "P-521", "-msg"])
+        has_hrr = any(x in output for x in ["HelloRetryRequest", "HRR", "hello_retry_request", "02 00 00"])
+        assert has_hrr
+        assert "ECH: accepted" in output or "02 79" in output
+
+    # --- SECTION 2: NEGATIVE TESTS (Security Audit) ---
+
+    def test_04_negative_no_ech_access(self):
+        """Verify standard clients are not granted ECH status."""
+        output = self.env.run_openssl(["-brief"])
+        assert "ECH: accepted" not in output
+
+    def test_05_negative_mismatched_public_name(self):
+        """Verify rejection when Outer SNI is incorrect."""
+        output = self.env.run_openssl(["-brief", "-servername", "wrong-gateway.com", "-ech_config_list", self.config])
+        assert "ECH: accepted" not in output
+        assert "ech_retry_configs" in output.lower() or "ech required" in output.lower()
+
+    def test_06_negative_corrupted_config(self):
+        """Verify rejection when the ECH key is invalid/poisoned."""
+        wrong_key = "AEH+DQA9tAAgACDG1DRKJzL4jbKU//fdPlSFfASYZgMrpthbvcsc+GbtKQAEAAEAAQAOeW91cmRvbWFpbi5jb20AAA=="
+        output = self.env.run_openssl(["-brief", "-ech_config_list", wrong_key])
+        assert "ECH: accepted" not in output
+        assert "ech required" in output.lower() or "0A0001A8" in output
+
+    def test_07_negative_tls_downgrade(self):
+        """Verify ECH is ignored if client attempts to use TLS 1.2."""
+        output = self.env.run_openssl(["-brief", "-tls1_2", "-ech_config_list", self.config])
+        assert "ECH: accepted" not in output
+
+    # --- SECTION 3: INTEROPERABILITY & LOGGING ---
+
+    def test_08_interoperability_grease(self):
+        """Verify server handles GREASE extensions gracefully."""
+        output = self.env.run_openssl(["-brief", "-ech_grease"])
+        assert "Verification: OK" in output
+        assert "ECH: accepted" not in output
+
+    def test_09_ech_environment_variables(self):
+        """Verify handshake triggers server-side logging of the ECH event."""
+        self.env.run_openssl(["-brief", "-ech_config_list", self.config])
+        logs = subprocess.check_output(self.env.docker_bin + ["tail", "-n", "20", "/usr/local/apache2/logs/error_log"]).decode()
+        assert any(x in logs for x in ["SSL handshake", "ECH", "ssl_engine"])

test/pyhttpd/ech/cases/test_01.py

@@ -0,0 +1,21 @@
+import pytest
+from lib.env import EchTestEnv
+
+class TestEchBrowser:
+
+    @pytest.fixture(autouse=True)
+    def setup_method(self):
+        self.env = EchTestEnv()
+        self.ech_config = self.env.get_ech_config()
+        if not self.ech_config:
+            pytest.fail("ECH Config not found in infrastructure/conf/ech/")
+
+    def test_01_browser_handshake_success(self):
+        """Test: Standard ECH Handshake via Firefox."""
+        page_source = self.env.run_browser(self.ech_config)
+        assert "ECH Decryption Successful" in page_source
+
+    def test_02_browser_with_grease(self):
+        """Test: ECH with GREASE enabled via Firefox."""
+        page_source = self.env.run_browser(self.ech_config, use_grease=True)
+        assert "ECH Decryption Successful" in page_source

test/pyhttpd/ech/cases/test_02.py

@@ -4,7 +4,6 @@
 import subprocess
 import re
 
-# Reuse your existing config and runner
 def get_latest_ech_config():
     path = "./conf/ech/ECH_key.pem"
     if not os.path.exists(path):

test/pyhttpd/ech/test_ech_performance.py …pyhttpd/ech/cases/test_03_performance.pytest/pyhttpd/ech/test_ech_performance.py renamed to test/pyhttpd/ech/cases/test_03_performance.py test/pyhttpd/ech/test_ech_performance.py renamed to test/pyhttpd/ech/cases/test_03_performance.py

@@ -0,0 +1,42 @@
+import subprocess
+import os
+import time
+import pytest
+from lib.env import EchTestEnv
+
+class TestServerRobustness:
+
+    @pytest.fixture(autouse=True)
+    def setup_method(self):
+        self.env = EchTestEnv()
+        self.config_path = "infrastructure/conf/ech/ECH_key.pem"
+
+    def test_01_missing_key_file(self):
+        """Requirement: Verify server handles missing ECH key file."""
+        os.rename(self.config_path, self.config_path + ".bak")
+
+        try:
+            subprocess.run(["docker-compose", "-f", "infrastructure/docker-compose.yml", "restart", "ech-server"])
+            time.sleep(2)
+
+            status = subprocess.check_output(["docker", "inspect", "-f", "{{.State.Running}}", "ech-server"]).decode().strip()
+
+            assert status == "false", "Server should not be running without its ECH key file"
+
+        finally:
+            if os.path.exists(self.config_path + ".bak"):
+                os.rename(self.config_path + ".bak", self.config_path)
+            subprocess.run(["docker-compose", "-f", "infrastructure/docker-compose.yml", "restart", "ech-server"])
+
+    def test_02_invalid_directive_syntax(self):
+        subprocess.run(["docker", "exec", "ech-server", "cp", "/usr/local/apache2/conf/httpd.conf.bak", "/usr/local/apache2/conf/httpd.conf"])
+        
+        subprocess.run(["docker", "exec", "ech-server", "sed", "-i", "s/SSLECHKeyDir/INVALID_COMMAND/", "/usr/local/apache2/conf/httpd.conf"])
+        
+        subprocess.run(["docker", "restart", "ech-server"])
+        time.sleep(2)
+
+        result = subprocess.run(["docker", "logs", "ech-server"], capture_output=True, text=True)
+        logs = result.stdout + result.stderr
+
+        assert "Syntax error" in logs or "Invalid command" in logs

test/pyhttpd/ech/cases/test_04_config.py

@@ -0,0 +1,67 @@
+import subprocess
+import os
+import re
+import time
+from selenium import webdriver
+from selenium.webdriver.firefox.options import Options
+
+class EchTestEnv:
+    def __init__(self):
+        self.connection_target = "localhost:443" 
+        
+        self.public_name = "localhost"
+        
+        self.private_host = "ech-test.fyp.local"
+
+        self.url = f"https://{self.private_host}"
+
+        self.docker_bin = ["docker", "exec", "ech-server"]
+        self.openssl_bin = "/opt/openssl-ech/bin/openssl"
+        self.ca_path = "/usr/local/apache2/conf/ssl/MyLocalCA.pem"
+
+    def get_ech_config(self):
+        path = "infrastructure/conf/ech/ECH_key.pem"
+        if not os.path.exists(path):
+            return "AEH+DQA9tAAgACDG1DRKJzL4jbKU//fdPlSFfASYZgMrpthbvcsc+GbtKQAEAAEAAQAOeW91cmRvbWFpbi5jb20AAA=="
+        with open(path, 'r') as f:
+            content = f.read()
+            match = re.search(r"-----BEGIN ECHCONFIG-----(.*?)-----END ECHCONFIG-----", content, re.DOTALL)
+            return match.group(1).replace("\n", "").strip() if match else ""
+
+    def run_openssl(self, args):
+        shell_cmd = (
+            f"LD_LIBRARY_PATH=/opt/openssl-ech/lib64 {self.openssl_bin} s_client "
+            f"-connect {self.connection_target} -servername {self.public_name} "
+            f"-CAfile {self.ca_path} -no_ticket " + " ".join(args)
+        )
+        cmd = self.docker_bin + ["sh", "-c", shell_cmd]
+        result = subprocess.run(cmd, input="Q\n", capture_output=True, text=True, timeout=10)
+        return result.stdout + result.stderr
+    
+    def run_browser(self, ech_config, use_grease=False):
+        """Apache-style Selenium wrapper."""
+        options = Options()
+        options.add_argument("-headless")
+        options.set_preference("network.proxy.type", 0)
+        options.set_preference("network.trr.mode", 0)
+        
+        # ECH enablement
+        options.set_preference("network.dns.echconfig.enabled", True)
+        options.set_preference("network.dns.local_echconfig", ech_config)
+        
+        # Trust and local environment settings
+        options.set_preference("security.enterprise_roots.enabled", True)
+        options.set_preference("network.http.ocsp.enabled", False)
+        
+        if use_grease:
+            options.set_preference("network.tls.grease.enabled", True)
+
+        driver = webdriver.Firefox(options=options)
+        driver.set_page_load_timeout(20)
+        
+        try:
+            driver.get(self.url)
+            time.sleep(2)
+            return driver.page_source
+        finally:
+            driver.quit()

test/pyhttpd/ech/lib/__init__.py

@@ -0,0 +1 @@
+NOT_A_KEY

test/pyhttpd/ech/lib/env.py

@@ -0,0 +1,52 @@
+-----BEGIN PRIVATE KEY-----
+MIIJQwIBADANBgkqhkiG9w0BAQEFAASCCS0wggkpAgEAAoICAQDO7Wh9nXlJ52OM
+olks0lRlH5jng3zo0R2Z7m8bqX8NdJszVmsw6zup+5JksC/I2dBr6tQ8qRHHxMgm
+oOulwIB7/bBbTpn8WqY7K96r5aQy+eey5zcA/8YIpJ8KXjc1GXyyx06aU4YxBdhJ
+AfOk7ySV7/ASdUjsHf4g39ua2amPlWhQzJtQGt5tlZm+KhtJgy6fWxWbiVQ3L/RJ
+Dktuuz9/GjvCPLuMwGX8g9rWtheQgXd6ihyaGrUo45CbQOHQZtfAYpdSrfuDamde
+UMgQxFUs4T9TS8PWxSqK0xxRs5m7jv/kSol7fKKrO7S8FSctpiVr3J9mC/yXJOpp
+FYxDfsgLRDHCIt3WsCNyA2NUvRtpY1NOq0ob72cA1sRXcvXXeXkM0aZnMSqyPwu2
+AEQRwtSj0b4Y+wIC+q0xXua5xFvvIBrkk2Mmuke8YmkcxXBM/nVqg31yqhRiuGRO
+slo9rRNjKyHvC4O+8MaXlkE8YCnN7E34bgqKngSh0SUvYlFt6ZQe26LpBBswF0eG
+gUf+xvGfi59VU0Vye8o+deC1C1NUgbgY3vMZLjdzVpW/WcQh32YS4ewAMJC2aM52
+h949zkUkw82z58PfKqqyoXSVkx5SZnaF4aVgtG0tE5f0/8odIUKUfHfDCHs8hen5
+Xo/en4npkEJU7G5D3XAHIvDy5UDXgQIDAQABAoICABGZP9yFFtJp/z2v9gkZl0tl
+aU3xUR+E33FexazS2McmbmeqlyG5M+EUUAJHuLyqh68R8Px6vZQhoIsmfvwhF9xT
+ulq9n9uGQyJ/q+evN2yNc/7zaqpnVmqYQ51wX14g/Ym/6Se3aE+FiXxGEfhqTVCC
+MEcFmg7Yyyr1Fvp/vgvD33QVvrTMoDOuOD3j21/AbCfp6XfJsXOjHLHU6SXw/2i6
+LLBrlWDWYSYdebBumqjz1dtCYUXa9SLV3c/ScBIXGQzX5bpGqUAnPcTX9nf0lrDj
+NE1LgYujx6c4Zq1tKqs4sXszOqeZtUT+ZjPj0anwejjG8fiOFuys21HWHxCDeRRT
+CNt1QvFeGf2tHXPR4p0YI0JK1CcBMmaHjnpycANCvbVEGCVUNvrRaDkVJM/dCQMd
+Kbx1HbQKF9gwrTvGGYYn/NNFVAmVrY63KCOamFyGbrChULi+A2nrS8y6IKj4Sx4D
+pKec6TNHknV/K2NfvXv+JCim5Zu6oYqZNiuWMm3shVF6Tt8pJ5eBoCpUpZgLwcBq
+TnpuGF8wHC37wl6J/AVpbsK0YAq9rxNrGYgCmsDfK+FhgSErLsCFaRs53T/x+Ryc
+RCUwk6gcy5ChMvJ7DzE9enwxHD6nq5sDe62FSp31sxsK2yudVdrhBfEsf70OZ+/W
+3XfYHbvq7CbRbEq/uqeRAoIBAQDv3UNhlbGhWb3YofN5KM5AJ0+DlKXDMMzu3cR5
+kBQN5263ljPj9aNnMXaJe8MulqHJm9VYXuKCB8v+rD6pIiOr/hiTghiTn5ef2Kom
+cZa6cQ3UfuyV5yzs2ljczbAntDf2Yeqz3jQwMthg+wuFOk8uiGTksJ6wGT/dPK8Q
+52QqrZNbyxiU+L+gtq42uW/P+3sXHih7lm/Xyl2eA3d8RxC0ClU8hnlbsoCFp7in
+X5tgh/dFKrBZEC76G+h+V1t5McImv+q3GEJm+UiTn3NTsmUcRP2mGEHo1J+WkUit
+5I45MEZGJaNYWX7Ey+SkHHL/6Q1lDyd89usecElobEIRHZ1ZAoIBAQDc2O70nNgu
+XAi90OVJNaz1P7/n8xtUtuj2VWzMknSJNNEJCfgPD9SfA0Zk3P91/egHD7bv03L4
+W83F2eSr8pqGy5HtDoqBxkouyuGTgK/p9lPjR8deT67ymnt+GrNM2fV3zKuXiCNo
+4AhkF9t9O/UG9B+76GOuBNzt2Dp8+kktPyqGVmXF31ccjCOc5oz5PorgGXpv6jNW
+mk/tPV5tmNBrQVdTh2hLahYjr4ufTu2WhXA7PBLeD+ltXBMbqKDNtQfQ+htqtZ3G
+7zRZqKSJxgzKJ4jfCl6bW55f+nG3FbE/T847LfeReabu1k6UT3R7N87NBMdsBJQd
+RxrKO5AdRf5pAoIBAQDMoTDowXImuo6xj4hMprk+FctJ77hyiuFqLpt9MaNKMVRN
+HsDqCxb55ELCC2l6B1vCyUT6/Qez8r7fZ0aVt+BCzKVewjABULdj0M1nuqPiLqyj
+yhw/zlaPQb9pr7hGRwMvGF3IURqou9fI9KLhZ9tBUW7xgpP+m6vWK/0WKLFVj3sV
+ZnB0NroUe4Sofw6ammpqUHos5SxJJgUz1rVKur3POrl4xyglSGVIoMtxTqkZcyVK
+Rp7nfFz3VnPDxPbur7p4oGW3CeUsQCLgfbk/gAOuWFUkK7Ge1jXHl+4vG7sRotNw
+6I8vwjnZ3jASqYqaM9IPkxwXCfePoi+d/C1ouKERAoIBAQCf1WUDpiwTSUqOTgxT
+csRtbqjuLxT9t69c8LBgUjKDRrVuzEc6Z2OjfdRJlWRRueRej/H/GlKgCpkfczY7
+d8Z8fgJrxdVaXO89dFnTzhQCyOMnn8BbsmHUdRehSaOwoCI2hOs/LSkrctC/2EBj
+H6yTTsVU0riprh1TCeYyo1WoqImXVhosHhrGr2nq2TT4AlqyG95v9tkW+XGVKpAX
+07wrk8umyV4jDnFdfGQZdR8gjAyQ4kZpbqyrGDNAFkfi+PziMtD65tx8qIyDwzjp
++WsyN3Cos7GK0MELh48bSVjRkGmajQcawyecvX97eRG9R8Okv6uwspObqOVrrbX8
+abbZAoIBAF71RtqyKWmvdqAyN1ZcnLNQ31SfqUh2ELMpL84d/WaK7pk/s4kVP5xe
+92qZBhkpKHxipnJOmeHqwjCO+PuA64M5eytC9bIYfrLTrXieub16fBNL2Dg1gCAh
+m0YlFLLweS4cNXOwvJXoOO5PTc4T63+yEwy8FxFf6WWFAenA5JzzqTBKZ7T4cTRb
+a6Tf3fm+tuQ2eexHZP9rZXz0gmAh9sec7xLSAIPmpf54nwPAs9FPB7X6fshRokPr
+yF8VHE1RnlMJ21rDCUTA9BboDRKBvYV6rVaj7H+SA90g87OL0LP5lvoG6G8fUejE
+uosCV96fm/Mvv5Vw/Ojs5krHq68VfHE=
+-----END PRIVATE KEY-----