Hmm, OP mentioned “Only edit what’s between parentheses” - don’t see anywhere that whole URL shouldn’t be changed…

Here’s a solution with perl (assuming you don’t want to change http/https after the start of ( instead of start of a line):

perl -pe 's/\[[^]]+\]\(\K(?!https?)[^)]+(?=\))/lc $&=~s|%20|-|gr/ge' ip.txt

• e flag allows you to use Perl code in the substitution portion.
• \[[^]]+\]\(\K match square brackets and use \K to mark the start of matching portion (text before that won’t be part of $&)
• (?!https?) don’t match if http or https is found
• [^)]+(?=\)) match non ) characters and assert that ) is present after those characters
• $&=~s|%20|-|gr change %20 to - for the matching portion found, the r flag is used to return the modified string instead of change $& itself
• lc is a function to change text to lowercase