The Java Security Exception is only one of several possible issues that can prevent the VNC handoff.
First, you will need to check that your software prerequisites are installed. The list for all supported i5/OS releases can be found here:
http://www-01.ibm.com/support/docview.wss?uid=swg21299738
Make sure the Java Group PTFs are up to date - it's not enough that they're installed; if they're too old, then problems may still persist.
Next, we'll need to check your TCP configuration.
1. Issue CFGTCP option 12
1a. Make sure your Host Name Search Priority is set to *LOCAL.
1b. Make a careful note of your hostname and domainname values.
2. Next issue CFGTCP option 10
2a. Check that the correct IP address lists your correctly spelled hostname and hostname.domainname.com, in that order. They should come first. Other things can be set to that IP, but they must be listed AFTER your system's host & domain values.
2b. Check that there's no ambiguity, that your hostname and hostname.domainname.com aren't set to multiple IPs.
3. Make sure there are no firewalls or popup blockers active between your installation PC and your iSeries.
- Make sure the Internet Explorer popup blocker is disabled
- Make sure the installation PC's Windows Firewall is taken down temporarily
- If possible, install from within your network so that you don't have the network firewall interfering. If you must install from outside the network firewall, make absolutely sure you have unrestricted access through the firewall, especially over ports 580x, 590x, and 2000.