It is important for you to improve your website, and if you are using a WordPress website it is easy for you to navigate and do it as long as you have the knowledge to implement that steps. 

So below are some important key factors that you are going to implement on your website, that will help you improve the loading speed of your site, as this is a crucial thing in terms of SEO.

• WP Plugins and Tools to USE, make sure you use plugins and get rid of those conflicts with any plugin that is not essential to your website.
• Image Compression, I think less than 50KB is the best, or maybe less than 100KB so improve and compress the size but avoid losing the quality.
• Cache, there are some tools or plugin that is useful to improve the speed with these the existing visitors
• will not wait to load all the information since it is stored already on the user's browser, and when they access it again it will load a lot faster.
• Hosting, if you consider or care about your website, and wanted to optimize your site in a Search Engine, it's always good to get decent hosting, it will help you get a faster loading, so think hosting as a computer the more RAM and CPU the faster it will get.
• CDN, yes this is best if you have a huge website like an images site, or maybe your site is an E-Commerce site, and as we all know 1 product has a lot of images, so consider also getting your own CDN, this is not really required for small business but its good to have. CDN needs a budget to avail of the service.
• Sprites - this will help shorten the number of requests and save bandwidth which of course helps your site loading speed improvement.