This paper extends the basic stochastic volatility (SV) model in order to incorporate the realized variance (RV) as an additional measure for the latent daily volatility. The particular model we use explicitly accounts for the dependency between daily returns and measurement errors of the realized volatility estimate. Within a simulation study we investigate the form of the dependency. In order to capture the long memory property of asset volatility, we explore different autoregressive dynamics for the latent volatility process, including heterogeneous autoregressive (HAR) dynamics and a two-component approach. We estimate the model using simulated maximum likelihood based on efficient importance sampling (EIS), producing numerically accurate parameter estimates and filtered state sequences. The model is applied to daily asset returns and realized variances of New York Stock Exchange (NYSE) traded stocks. Estimation results indicate that accounting for the dependency of returns and realized measures significantly affects the estimation results and improves the model fit for all autoregressive dynamics.