I did that in my development environment and it seems a restart of the application is required to take effect of the changes. Just let me know if a restart is really required or if i missed something.
After making changes in the dynamic attributes configuration, click Apply settings button in the categories browser. Changes can also be applied via Administration > JMX Console by calling the clearDynamicAttributesCache() method of the app-core.cuba:type=CachingFacade JMX bean.