What is a Trusted Certificate?
Starting from Tableau Server 2020.2 analytics extensions are configured with admin UI and not with TSM (more details here – Multiple Analytics Extensions Connections with Tableau Server 2020.2). And if a connection is secured (Require SSL checkbox is on) Tableau will validate certificate used by TabPy, Rserve or any other analytics extensions.
For the certificate to be trusted there are a few checks: it should be issued to the host where the analytics extensions is running, dates (valid after/before) are valid, the certificate is signed with another certificate Tableau Server trusts and some other checks. But what certificates are trusted?
Each OS ships with preinstalled trusted certificates and anything signed with any of those trusted certificates is trusted (considering all other checks mentioned above pass). Each OS has its own way of installing a trusted certificate – refer to your OS documentation.
Leaf VS Whole Chain Certificate
When analytics extension sends it certificate to Tableau it can either send the whole chain (including all the certificates it signed with) or just the very last in the chain certificate (leaf certificate).
For example, we have certificate issues for machine my-server-cert which is signed with my-org-mid-cert, which is signed with my-org-root-cert:
NOTE: number of mid-certificated can be any, but there could be rules on how deep the chain can be to not be rejected.
TabPy, if uses my-server-cert from the example above sends to Tableau the whole chain.
If the my-server-cert is installed on Tableau Desktop or Server machine as trusted – validation for it passes. Otherwise, if my-org-mid-cert is trusted – my-server-cert is trusted as well. And finally, if my-org-root-cert is trusted – my-org-mid-cert and my-server-cert are trusted. This means it is sufficient to install my-org-root-cert as trusted on the Tableau machine to make the whole chain trusted.
Rserve, when configured to use the same certificate as on the example above only sends to Tableau leaf cert – meaning only my-server-cert. This means for Tableau to trust the certificate my-server-cert has to be installed as trusted.
Self-signed are certificates which are not signed with any other certificate.
For self-signed certificates to be trusted they need to be installed on the client machine (machine which runs Tableau Desktop or Server).