Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added datasource Bank of England database #85

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions docs/src/downloads.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# Downloading from remote data source

Three eponymous methods are provided for downloading free data, from
Yahoo Finance (yahoo) the Federal Reserve St. Louis (fred) and the
United Kingdom's Office for National Statistics (ons)
Yahoo Finance (yahoo) the Federal Reserve St. Louis (fred), the
United Kingdom's Office for National Statistics (ons) and the
Bank of England's database (boe).

These methods take a string argument that represents the name of the
desired data set to be downloaded.
Expand Down Expand Up @@ -35,4 +36,14 @@ This is the ONS' headline monthly price inflation statistic.

```@docs
ons
```
## BOE

For the `boe()` method, the default
seriescode is the `IUDSOIA`, which
represents the daily Sterling Overnight Index Average
which is one of the 10 most popular BoE timeseries.

```@docs
boe
```
3 changes: 2 additions & 1 deletion src/MarketData.jl
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ export AAPL, BA, CAT, DELL, EBAY, F, GE, TX,
# downloads.jl
export AbstractQueryOpt,
YahooOpt,
yahoo, fred, ons
BoeOpt,
yahoo, fred, ons, boe

###### include ##################

Expand Down
120 changes: 120 additions & 0 deletions src/downloads.jl
Original file line number Diff line number Diff line change
Expand Up @@ -193,3 +193,123 @@ function ons(timeseries::AbstractString = "L522", dataset::AbstractString = "MM2
end
TimeArray(ta, meta = json["description"])
end

"""
struct BoeOpt <: AbstractQueryOpt
Datefrom # the start time
Dateto # the end time
UsingCodes # indicate using series codes
CSVF # "TT" (Tabular with titles), "TN" (Tabular no titles), "CT" (Columnar with- titles) or "CN" (Columnar no titles)
VPD # provisional data is required
end

The Bank of England Database API query object.

# Examples

```jl-repl
julia> t = Dates.today()
2022-04-06

julia> BoeOpt(Datefrom = t - Year(2), Dateto = t)
BoeOpt with 5 entries:
:Datefrom => "06/Apr/2020"
:Dateto => "06/Apr/2022"
:UsingCodes => "Y"
:CSVF => "TN"
:VPD => "Y"
```
"""
struct BoeOpt <: AbstractQueryOpt
Copy link
Member

@Arkoniak Arkoniak Apr 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually we use CamelCase for modules and type names, not for fields.
Can we switch to lowercase? Like

struct BoeOpt
    date_from::Date
    date_to::Date
    using_codes::Bool
    csvf::String
    vpd::Bool
end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An AbstractQueryOpt is passed to the HTTP GET command with its fieldnames and values as a query.

I chose fieldnames to be consistent with the API documentation. https://www.bankofengland.co.uk/boeapps/database/Help.asp#CSV

However, I agree that lowercase is standard for fieldnames, and since the parameter names in a HTTP GET query are case-insensitive this shouldn't cause an issue. However, underscores would not be consistent with AbstractQueryOpt as I would need to change the field names to construct the query.

Datefrom::Date
Dateto::Date
UsingCodes::Bool
CSVF::String
VPD::Bool

BoeOpt(; Datefrom::Date = Date(1963, 1, 1),
Dateto::Date = Dates.today(),
UsingCodes::Bool = true,
CSVF::String = "TN",
VPD::Bool = true
) =
new(Datefrom, Dateto, UsingCodes, CSVF, VPD)
end

function Base.iterate(opt::BoeOpt, state = 1)
(state > length(BoeOpt)) && return nothing
k = fieldname(BoeOpt, state)
v = getfield(opt, state)
v′ = v isa Date ? Dates.format(v,dateformat"dd/u/yyyy") : v isa Bool ? v ? "Y" : "N" : v
(k => v′, state + 1)
end

"""
boe(seriescode::String="IUDSOIA")::TimeArray

The boe() method is a wrapper to download financial and economic time series data from the Bank of England (BoE) database.

The boe() method takes a string argument that corresponds to a series code from the BoE database.
It returns the data in the TimeSeries.TimeArray data structure. When no argument is provided, the
default data set is the daily Sterling Overnight Index Average (SONIA) rate.

# Examples

```julia
GBPUSD = boe("XUDLGBD")
SONIA = boe()
```

```jl-repl
julia> start = Date(2018, 1, 1)
2018-01-01

julia> boe(:IUDSOIA, BoeOpt(Datefrom = start))
1078×1 TimeArray{Float64, 1, Date, Vector{Float64}} 2018-01-02 to 2022-04-04
│ │ IUDSOIA │
├────────────┼─────────┤
│ 2018-01-02 │ 0.4622 │
│ 2018-01-03 │ 0.4642 │
...

julia> boe("XUDLGBD,XUDLERS", BoeOpt(Datefrom = start))
1079×2 TimeArray{Float64, 2, Date, Matrix{Float64}} 2018-01-02 to 2022-04-05
│ │ XUDLGBD │ XUDLERS │
├────────────┼─────────┼─────────┤
│ 2018-01-02 │ 0.7364 │ 1.1274 │
│ 2018-01-03 │ 0.74 │ 1.124 │
...
```

# References

https://www.bankofengland.co.uk/boeapps/database/
https://www.bankofengland.co.uk/boeapps/database/Help.asp#CSV
https://www.bankofengland.co.uk/statistics/details

# See Also

- yahoo() which is a wrapper to download financial time series for stocks from Yahoo Finance.
- fred() which accesses the St. Louis Federal Reserve financial and economic data sets.
- ons() which is a wrapper to download financial and economic time series data from the Office for National Statistics (ONS).
"""
function boe(seriescodes::AbstractString = "IUDSOIA", opt::BoeOpt = BoeOpt())
url = "http://www.bankofengland.co.uk/boeapps/iadb/fromshowcolumns.asp"
parameters = Dict(
"csv.x" => "yes",
"SeriesCodes" => seriescodes
)
res = HTTP.get(url, query = merge(parameters,Dict(opt)))
@assert res.status == 200
if Dict(res.headers)["Content-Type"] != "application/csv"
if Dict(res.headers)["Content-Type"] == "text/html"
mat = match(r"<p class=\"error\">\s+([\w\s]+)",String(res.body))
!isnothing(mat) && throw("Error message from BoE: $(mat.captures[1])")
end
throw("CSV not returned from BoE, it's likely that the SeriesCodes, ($seriescodes) are invalid")
end
csv = CSV.File(res.body, dateformat=dateformat"dd u yyyy")
sch = TimeSeries.Tables.schema(csv)
TimeArray(csv, timestamp = first(sch.names)) |> cleanup_colname!
end
boe(s::Symbol, opt::BoeOpt = BoeOpt()) = boe(string(s), opt)
8 changes: 8 additions & 0 deletions test/downloads.jl
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,12 @@ using Test
ta = ons()
@test ta |> timestamp |> length > 100
end

@testset "BOE" begin
t = Dates.today() - Year(2)
opt = BoeOpt(Datefrom = t)
ta = boe("XUDLGBD",opt)
@test ta |> timestamp |> length > 100
@test timestamp(ta)[1] >= t
end
end