re: Debunking Kimball Effective Dates
I agree with John about focusing on a robust ETL process instead of trying to minimise the data redundancy at price of less clarity on physical model, although I found the debates around the topic are...
View Articlere: Debunking Kimball Effective Dates
so I attempted to build this as a POC for a client who is experiencing this overlapping type2 issue. My tables in question are relatively small (under 400k records), so VIEW performance does not scare...
View Articlere: Debunking Kimball Effective Dates
"I think the author did not understand Kimball in First place"Well you're entitled to your opinion. As am I. I think I understand Kimball's prescriptive methods pretty well and am well qualified to...
View Articlere: Debunking Kimball Effective Dates
I think the author did not understand Kimball in First place. The date ranges in the example are overlapping and that is not correct. You can not have a record ending on 27/11/2009 and next record...
View Articlere: Debunking Kimball Effective Dates
If you want IsCurrent instead of IsLatest, you could do something like:, CONVERT(BIT,CASE WHEN GETDATE() BETWEEN MIN(dim1.[SCDStartDate]) AND MIN(dim2.[SCDStartDate]) THEN 1 ELSE 0 END) AS...
View Articlere: Debunking Kimball Effective Dates
And this is one of the reasons I am giving up on Kimball approaches. The "Fact & Dimension table" model only really works for immutable historical data about point-in-time events (one of the...
View Articlere: Debunking Kimball Effective Dates
So Jamie, curious does this approach work when dealing with large dimension table and using Hive or Impala? Seems like the join may be too expensive...
View Articlere: Debunking Kimball Effective Dates
Hihave a little case here that I hope to solve using Your code With a twistHere is the case:I am collecting data from several Sources about Companies, but let us a dummy exampleI can get several rows...
View Articlere: Debunking Kimball Effective Dates
Kimball usually leaves implementation details up to the implementer. I would simply say you've demonstrated one implementation approach to accomplish the same end goal. I've never implemented a DW...
View Articlere: Debunking Kimball Effective Dates
My thinking is that, in a data warehouse, row level constraints (ie: insuring that two rows for the same product or customer don't have overlapping effective periods) and computed columns (ie: deriving...
View Articlere: Debunking Kimball Effective Dates
Jamie is absolutely right... we could introduce a contradiction if we redundantly store a point in time twice, one to represent the end of an interval, and then again to represent the start of the next...
View Articlere: Debunking Kimball Effective Dates
This is a pretty dated article, but the concept is fundamental to traditional data warehousing. I was browsing around and felt like I should comment on this for anyone new looking around for an...
View Articlere: Debunking Kimball Effective Dates
I may be a bit late into this conversation however I noticed the issue as soon as I looked at the End date of the first record and the start date of the second record. The dimension records in SCD II...
View Articlere: Debunking Kimball Effective Dates
It's always nice to run into a five year old blog post with an ongoing discussion!I have a few points to add. For one, I like using the BETWEEN join (when a surrogate key look-up is unavailable, for...
View Articlere: Debunking Kimball Effective Dates
Hi PMerrill,Thanks for the comment."I'm a big fan of questioning our assumptions, so thanks for this conversation"I'm glad at least one person out there appreciates it :)"what about the ETL Queries?...
View Articlere: Debunking Kimball Effective Dates
I'm a big fan of questioning our assumptions, so thanks for this conversation."The primary purpose of SCD start/end dates is not for end user querying, it is for the purposes of doing lookups during...
View Articlere: Debunking Kimball Effective Dates
Hi Ivan,Thanks for the comment."I think what has been lost in all this discussion is that the purpose of the start/end dates is to make QUERYING the data easier."I disagree. The primary purpose of SCD...
View Articlere: Debunking Kimball Effective Dates
I think what has been lost in all this discussion is that the purpose of the start/end dates is to make QUERYING the data easier. a one-time hit during ELT (if there is a performance hit) is...
View Articlere: Debunking Kimball Effective Dates
Oops - just realized - no DEFAULT required on the RowVer field as it gets handled by the insert logic. :)
View Articlere: Debunking Kimball Effective Dates
Hi folks,Just thought I'd post in an alternative approach which guarantees the correct dates, and also provides a row-versioning metric. For the record - this approach is optimized for processing the...
View Article