There is no way to add Contacts folders from mailboxes shared with you.

It works with Public Folder contacts and SharePoint Lists. You can see the list of Address Books when clicking To, From, or CC buttons when composing a message. It can be useful to have Public Folder or SharePoint Contacts to choose from.

AdressBook

To add Public Folder Contacts folder to the list of Address Books, you need to do the following:

  1. Right-click the Public Folder Contacts.
  2. Click Properties.
  3. Click Outlook Address Book tab.
  4. Check the box Show this folder as an e-mail Address Book.

Steps

Note: Only contacts with email addresses assigned will be added to Outlook Address Book. Contacts without email addresses will not be shown.