We study a vector dark matter (VDM) model in which the VDM couples to the Standard Model sector via the Higgs portal. As a result, the VDM can be produced via the freeze-in mechanism. In particular, we find that the electroweak phase transition have a great impact on the prediction of the VDM relic density. We further assume that the dark Higgs which gives the VDM mass is so light that it can induce strong VDM self-interactions and solve the small-scale structure problems in the Universe. Due to the extreme smallness of the Higgs portal coupling derived by the freeze-in mechanism, the dark matter direct detection bounds do not limit our model at all, which is illustrated with the latest LUX data. However, the model is well constrained by the indirect detections of VDM from BBN, CMB, AMS-02, and diffuse $\gamma$/X-rays. Consequently, only when the dark Higgs mass is at most of ${\cal O}({\rm keV})$ can we find some parameter region that leads to a right amount of VDM relic abundance and an appropriate VDM self-scatterings while satisfying all other constraints simultaneously.