Admin Security

I have htaccess set to deny from all allow from my ip, chmod to 644 and changed folder name.

Is this the correct security configuration for my admin folder and files. I can still access the admin login portal from anywhere. If I chmod to only allow access from the user would this be much safer.