I'm not sure if I ever mentioned this or not, but I'm terrible with plants. They tend to die on me. Period. I either overwater them, underwater them, don't give the enough sunlight or they just wither away and die for no apparent reason at all. What can I say? I was not born with a green thumb. A few weeks ago, I got a tiny olive tree as a surprise present, and logically, I worry about it, mostly because the weather here is less than stellar at the moment and sunlight seems far off. So, I have decided to consult olivetreegrowers.com for some good advice about keeping a potted olive tree in my neck of the woods.

Can olives be grown in pots?
Yes, but they will require more attention and care than will trees planted in the ground. Most people use a pot that is too small and do not water sufficiently. To keep any tree in a pot for long requires a really big pot. Olive trees can make fruit while growing in pots, but keep in mind that if your olive tree does not put on new growth, you will have little or no fruit the following year. Your olive tree will continue to need light and water if you bring in indoors for the winter; a bright, cool spot out of heater drafts is preferred. 

Can they be grown in cold climates?
Superficial damage to olive trees begins when temperatures fall into the low 20s. Significant damage occurs by mid-teens. Single digit temperatures are fatal. Olive trees (especially semi-dwarf cultivars) are easily grown in containers for bringing indoors for protection in winter. We suggest bringing your tree indoors when temperatures dip to below freezing, and taking it back outside on mild days whenever possible.

Any more tips on winter care? 
  • Leave container-grown olive trees outside in a sunny location until temperatures fall below freezing.
  • Remember that trees in pots will be more vulnerable to cold than trees in the ground.
  • When sub-freezing temperatures are expected, bring potted olive trees indoors to a cool, well-lighted place. A room with a south-facing windows and a temperature of 40-50 degrees is ideal.
  • Move it in and out. When temperatures are above 40 degrees, move your tree outside to a sunny location during the day. Move it back inside if freezing is expected overnight.
  • Water. Remember that your tree will need some water during the winter, too. Keep the soil slightly moist through and through. Check the top of the soil and drain holes to be sure soil is not just damp on the top.

What pests do they get?
In over 20 years of growing olives, the only significant pest we have found is an armored scale insect. Adult scales look like 1/2 a BB and are usually on the shadier parts of the tree (e.g. undersides of the leaves). The trees you receive from us will be pest-free, but you should monitor your tree for pests.
If scale insects are found they may be controlled by a number of products, both organic and not. If you have only one or a few trees the simplest control is manual — remove them with your hands. If there are ants on your tree, it may be because they are eating the droppings of scale insects. Untreated scale infestations may lead to the development of sooty mold.

How do I (re)pot the olive tree?
When you are ready to re-pot your olive tree, get a larger pot and some good quality commercial potting mix. Always be sure to use potting "soil" that drains well. Never use dirt out of your yard or heavy, dense potting medium labeled for house plants. It is good to go ahead and move it up and let it feed if the weather is warm. Unless you are in a very warm climate, don't fertilize it after August. Always be sure to use fertilizers that are safe for putting in pots. We like Osmocote Plus with minor elements. It's expensive but worth it.
Be careful with your tree — don't pull it out by its stem. You may want cut the pot for easy removal. If you want to reuse the pot, place one hand over the top of the root ball to hold the potting medium in place, turn the tree upside down (or nearly), and give it a quick shake or two to loosen it from the pot. Remove the pot and upright the tree, supporting it by the bottom of the root ball. Never tear apart the roots or wash the potting medium out of the root ball. Have the new home ready before taking your tree out of its pot. You can measure the potting medium level by firming some in the new pot and then placing the still potted tree in it to see that the level is good. We recommend against adding rocks or other material in the bottom of the pot because it robs your tree of growing medium. Pebbles, pot shards and the like can also block drain holes. Be sure to water it often after the transition, and regularly thereafter.
How much water does my potted olive tree need?
While MATURE trees can resist some drought, olives are not desert plants, and can suffer or even die from lack of water. Trees kept in containers need more water than those planted in the ground. Do not let the bottom of the pot stand in water.

Do I need to prune my tree?
Olive trees may be pruned to the desired shape. Some people prefer a tree form; others like a rounded shrub as is done in production groves to facilitate picking of the fruit. Pruning between mid-February and the ripening of fruit in the fall, except for the lightest tipping of new shoots, can result in a reduced crop. Otherwise, prune to the desired shape. Keep in mind that olives bear their fruit on last year's new growth.
Alright, that's a lot to take it. I think it come to the following, though: olive trees are easy plant to handle, but they are harder to keep alive in a pot. As soon as the weather allows, take them outside for some much needed sunlight. Handle with care. If you need to take them inside, keep them away from the heater, but put them in a sunny place. Give them enough water, wherever they are, but don't drown them. They don't like it if water remains behind in their saucer. For easy to handle trees, they sure have a lot of wishes. Sill, enough water and enough sunlight are things even I can provide. If my tree survives until next year, I'll take a look at that pruning thing again. Wish me luck!

Information taken from: here, here, here, and here.