SSLError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url

[sijiashen]

Issue
I ran encoding = tiktoken.get_encoding("cl100k_base") and encountered the following error:
SSLError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1129)')))

Cause & Solution
Our hypothesis is that we are not allowed to making a request to a host outside of the kubeflow VPC.
Q1: Would you say this is the primary cause?

If the answer to Q1 is yes:
Q2: Does tiktoken allow us to read from a file system artifact instead of making a web call?

[polinabee]

I have the same issue, did you find any resolution?

[anithamudigoudar]

I have the same issue, is there any update on this?

[polinabee]

I seem to be able to get rid of this issue by setting a more restrictive SSL token in openssl dir for my python

[anithamudigoudar]

@polinabee Can you please provide the steps to do it please

[songwang41]

I also encountered this error? any solution?

[nkilm]

It's because of the network issue. You can manually download the .tiktoken file and specify the location of this file in tiktoken using TIKTOKEN_CACHE_DIR environment variable.

You can find a detailed guide here - https://stackoverflow.com/questions/76106366/how-to-use-tiktoken-in-offline-mode-computer

[wirapratamaz]

It's because of the network issue. You can manually download the .tiktoken file and specify the location of this file in tiktoken using TIKTOKEN_CACHE_DIR environment variable.

You can find a detailed guide here - https://stackoverflow.com/questions/76106366/how-to-use-tiktoken-in-offline-mode-computer

Hi man how can I fix the SSLError with self-signed certificates when trying to use tiktoken offline? Also, what's the best way to make sure the files it needs are cached properly?

[nkilm]

It's because of the network issue. You can manually download the .tiktoken file and specify the location of this file in tiktoken using TIKTOKEN_CACHE_DIR environment variable.
You can find a detailed guide here - stackoverflow.com/questions/76106366/how-to-use-tiktoken-in-offline-mode-computer

Hi man how can I fix the SSLError with self-signed certificates when trying to use tiktoken offline? Also, what's the best way to make sure the files it needs are cached properly?

You can download and keep the .tiktoken file in your project itself and set TIKTOKEN_CACHE_DIR to where the .tiktoken file resides.