<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Jim,</p>
<p>Census is a large database. Suggest you start by looking at PDF's
of the actual census questionnaire <b>booklets</b>. In my mind
that is the easiest way to get a oriented to exactly what is and
isn't covered. You then need to skim/look at <b>methodology </b>to
understand the samples, the sampling frequencies, and the masking
they apply to protect confidentiality of individual information.
(Much better masking than big data!)</p>
<p>Personally, I like the <i>Tiger boundary files</i> to get the
polygons (block, block group, tract, county, zip code...) Note
that boundaries change pretty frequently, you need to line up the
correct version by year. <br>
</p>
<p>Block is the most granular. The more granular the reporting the
greater the number of completed surveys they need (and the cost
for that goes up). Pretty sure that data at the block level is
100% sampled ONLY every 10 years and very few data elements are
actually included in that. (see the actual pdf of that level of
the survey.) Remember too even though they have the data, not much
will be shared with you at the block level). After that they
survey (just SAMPLES) at the block group level, on a rotating <i>multiple
year basis</i>, and use ample inference.</p>
<p>American Fact Finder is the database name & homegrown query
language you use to get to the census database and pluck out a
slice. <br>
<a class="moz-txt-link-freetext" href="https://www.census.gov/programs-surveys/sis/resources/data-tools/aff.html">https://www.census.gov/programs-surveys/sis/resources/data-tools/aff.html</a>
The UI is html forms based, clunky, but usable with some practice.
After writing your query via html forms, you run it, and then it
spits back your data in a download file. <br>
</p>
<p>Virtually all the census data available at Fact Finder is pure
relational except for the boundary (Tiger) files. My personal
philosophy - do as much as you can with your relational database
of choice (or spreadsheet if it's small and simple) prior to
looking at the mapping (spatial) aspects. Microsoft Excel w/ Power
BI tools is pretty good at ETL, and its Power Query compresses the
data as it goes into RAM/Swapped memory. It can then spin that
compressed data ball pretty well (I think by only uncompressing
operative rows or columns, as needed on the fly). So, even on a
modest desktop machine you can handle quite a bit . The trick is
to ETL only what you really need vs. trying to recreate the entire
census DB locally as step one. <br>
</p>
<p>Since there tend to be many steps in the ETL tool chain strongly
suggest you <b>keep good notes </b>and "breadcrumbs" along the
trail, so you can repeat it in six months if you have that need.
What seems "so obvious and unforgettable" just after you work all
through it, magically evaporates from the memory in about 90 days,
or less. <br>
</p>
<p>If you look at your time and effort dealing with it all you may
find 3rd party forecasting firms better able at keeping up with
the constant survey frequency and detailed changes, and providing
enhanced current projections at the granularity you may need.
However at other times it pays to dive in yourself so you know
just how statistically flimsy some of those "enhanced" projections
actually are (both CENSUS estimates and 3rd party "enhanced"
estimates).</p>
<p>The data is only made "accurate enough" to serve the general
needs of the government plus or minus. If your application
requires more pure / more recent data you may have to find ways to
supplement census data to get a better focus on the situation. <br>
</p>
<p>Its all available (except for the masking), just takes some
digging. Use the questionnaires up front as your "treasure map". <br>
</p>
<p>Rick</p>
<p>PS Survey of Current <i>Business </i>is it's own can of worms.
Operations and plants vs. corporations and subsidiaries, economic
vs. financial accounting, masking at the firm and industry
level.... takes a while to unravel all that. Try to find "high
tech payroll" last quarter vs 4 quarters ago by county or zip....
Only IRS or State Disability operations have that data...and they
are not sharing much! Yes they have the individual's information -
payroll, occupational classification, etc, but relate it, mask it
and report it? "Not on our budget"....<br>
</p>
<div class="moz-cite-prefix">On 5/28/2019 12:52 PM, Thomas, Jim
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:MWHPR1101MB2333BDA4647CD59FF32CC5A39F1E0@MWHPR1101MB2333.namprd11.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<style><!--
/* Font Definitions */
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1027" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Hi All, what is the best data source for US
Census GIS-formatted data? Ideally, I’m looking for a source
where I can extract data based on a geographic area such as
block, block group, tract, county, or user-defined area. I
need to report population data based on custom buffers and
identify feature data such as institutions, parks, etc. within
those buffers. In the past, I’ve downloaded the data from
Esri and extracted it myself, but it’s a lot of data and takes
time to query. Additionally, I’ve use BAO to run the
population reports for the study area, but the maps from that
service lack the detail we need. That’s why we create custom
maps using the census data. Our sites can potentially be
anywhere in the US. Suggestions?
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks, <o:p></o:p></p>
<p class="MsoNormal">Jim.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:8.0pt"><b><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#43B02A">Jim
Thomas, PG, GISP</span></b><b><span
style="font-size:10.0pt;color:#43B02A"><br>
</span></b><i><span
style="font-size:9.0pt;font-family:"Arial",sans-serif;color:black">Senior
Project Geoscientist</span></i><i><span
style="font-size:9.0pt;color:black"><br>
</span></i><b><span style="font-size:10.0pt;color:#43B02A"><br>
</span></b><!--[if gte vml 1]><v:shapetype id="_x0000_t75" coordsize="21600,21600" o:spt="75" o:preferrelative="t" path="m@4@5l@4@11@9@11@9@5xe" filled="f" stroked="f">
<v:stroke joinstyle="miter" />
<v:formulas>
<v:f eqn="if lineDrawn pixelLineWidth 0" />
<v:f eqn="sum @0 1 0" />
<v:f eqn="sum 0 0 @1" />
<v:f eqn="prod @2 1 2" />
<v:f eqn="prod @3 21600 pixelWidth" />
<v:f eqn="prod @3 21600 pixelHeight" />
<v:f eqn="sum @0 0 1" />
<v:f eqn="prod @6 1 2" />
<v:f eqn="prod @7 21600 pixelWidth" />
<v:f eqn="sum @8 21600 0" />
<v:f eqn="prod @7 21600 pixelHeight" />
<v:f eqn="sum @10 21600 0" />
</v:formulas>
<v:path o:extrusionok="f" gradientshapeok="t" o:connecttype="rect" />
<o:lock v:ext="edit" aspectratio="t" />
</v:shapetype><v:shape id="Picture_x0020_2" o:spid="_x0000_s1026" type="#_x0000_t75" alt="Golder_Logo_Sig" style='position:absolute;margin-left:0;margin-top:0;width:52.55pt;height:47.1pt;z-index:251659264;visibility:visible;mso-wrap-style:square;mso-width-percent:0;mso-height-percent:0;mso-wrap-distance-left:0;mso-wrap-distance-top:0;mso-wrap-distance-right:9pt;mso-wrap-distance-bottom:0;mso-position-horizontal:left;mso-position-horizontal-relative:text;mso-position-vertical:top;mso-position-vertical-relative:line;mso-width-percent:0;mso-height-percent:0;mso-width-relative:page;mso-height-relative:page' o:allowoverlap="f">
<v:imagedata src="mailbox:///C:/Users/rick/AppData/Roaming/Thunderbird/Profiles/ia5v1taz.default/Mail/mail.clbcm.com/Inbox.sbd/B%20priority.sbd/FL.sbd/shrug-l?number=3349804&header=quotebody&part=1.1.2&filename=image001.jpg" o:title="Golder_Logo_Sig" />
<w:wrap type="square" side="right" anchory="line"/>
</v:shape><![endif]--><!--[if !vml]--><img
style="width:.7333in;height:.65in"
src="cid:part1.54397664.2BB2B52E@clbcm.com"
alt="Golder_Logo_Sig" v:shapes="Picture_x0020_2" class=""
width="70" hspace="6" height="62" align="left"><!--[endif]--><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">Golder
Associates Inc.</span><span
style="font-size:10.0pt;color:black"> <br>
</span><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">9428
Baymeadows Road, Suite 400, Jacksonville, Florida, USA 32256</span><span
style="font-size:10.0pt;color:black">
<br>
</span><b><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#43B02A">T:
</span></b><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">+1
904 363-3430 |
</span><b><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#43B02A">D:
</span></b><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">+1
904 421-4254 |
</span><a name="web" moz-do-not-send="true"></a><a
href="http://www.golder.com/" moz-do-not-send="true"><span
style="mso-bookmark:web"><b><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#43B02A">golder.com</span></b></span><span
style="mso-bookmark:web"></span></a><b><span
style="font-size:10.0pt;color:#43B02A">
<br>
</span></b><a name="linkedin" moz-do-not-send="true"></a><a
href="https://www.linkedin.com/company/golder/"
moz-do-not-send="true"><span style="mso-bookmark:linkedin"><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#43B02A;text-decoration:none">LinkedIn</span></span><span
style="mso-bookmark:linkedin"></span></a><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#43B02A">
| <a name="instagram" moz-do-not-send="true"></a><a
href="https://www.instagram.com/golderassociates/"
moz-do-not-send="true"><span
style="mso-bookmark:instagram"><span
style="color:#43B02A;text-decoration:none">Instagram</span></span><span
style="mso-bookmark:instagram"></span></a> |
<a name="facebook" moz-do-not-send="true"></a><a
href="https://facebook.com/golderassociates/"
moz-do-not-send="true"><span style="mso-bookmark:facebook"><span
style="color:#43B02A;text-decoration:none">Facebook</span></span><span
style="mso-bookmark:facebook"></span></a> |
<a name="twitter" moz-do-not-send="true"></a><a
href="https://twitter.com/GolderAssociate/"
moz-do-not-send="true"><span style="mso-bookmark:twitter"><span
style="color:#43B02A;text-decoration:none">Twitter</span></span><span
style="mso-bookmark:twitter"></span></a></span><span
style="font-size:10.0pt;color:#43B02A"><br>
<b><br>
</b></span><b><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#43B02A">Work
Safe, Home Safe</span></b><b><span
style="font-size:10.0pt;color:#43B02A">
<br>
<br>
</span></b><i><span
style="font-size:7.5pt;font-family:"Arial",sans-serif;color:gray">This
email transmission is confidential and may contain
proprietary information for the exclusive use of the
intended recipient. Any use, distribution or copying of
this transmission, other than by the intended recipient,
is strictly prohibited. If you are not the intended
recipient, please notify the sender and delete all copies.
Electronic media is susceptible to unauthorized
modification, deterioration, and incompatibility.
Accordingly, the electronic media version of any work
product may not be relied upon.</span></i><span
style="font-size:7.5pt;color:gray">
<br>
</span><b><span style="font-size:10.0pt;color:#43B02A"><br>
</span></b><span
style="font-size:7.5pt;font-family:"Arial",sans-serif;color:gray">Golder
and the G logo are trademarks of Golder Associates
Corporation</span><span style="font-size:7.5pt;color:gray">
<br>
</span><b><span style="font-size:10.0pt;color:#43B02A"><br>
</span></b><b><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:gray">Please
consider the environment before printing this email.</span></b><span
style="font-size:10.0pt;color:gray">
</span><b><span
style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#43B02A"><o:p></o:p></span></b></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
SHRUG-L mailing list
<a class="moz-txt-link-abbreviated" href="mailto:SHRUG-L@lists.dep.state.fl.us">SHRUG-L@lists.dep.state.fl.us</a>
<a class="moz-txt-link-freetext" href="http://lists.dep.state.fl.us/mailman/listinfo/shrug-l">http://lists.dep.state.fl.us/mailman/listinfo/shrug-l</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
Richard J. Labs, CFA, CPA
CL&B Capital Management, LLC
Phone: 315-637-0915
E-mail (preferred for efficiency): <a class="moz-txt-link-abbreviated" href="mailto:rick@clbcm.com">rick@clbcm.com</a>
3213 Yorktown Dr, Tallahassee, FL 32312-2015
June-August: 408B Holiday Harbour, Canandaigua, NY 14424</pre>
</body>
</html>