Map Compression .. a test

GSV Three Minds in a Can said:

(polling Paul, if he's out there) .. After the discussions of .jpg vs
.gif (or .bmp or .TIF or whatever) for compressing and saving maps, I've
set up a blind test sample at:

www.quik.clara.co.uk/samples.tif

, which contains a 'stitch up' of two .gifs (150 bpi and 400 bpi - 1.7Mb
and 9.9 Mb respectively) and two .jpgs (400 bpi and 800 bpi - 3.9 and
17.3Mb). The sizes are for a 'full scanner page' of map (1947 OS
sheet129, as it happens).

Your challenge, should you choose to accept it (it's a 1.7Mb download,
as a .tif file) is to decide which of the 4 segments was which, and why
the .gif beats the .jog, give that (at 400 bpi) it's twice as big. or
maybe I used the wrong .gif production program?

I can actually see more artefacts in the .gif (random lighter speckles)
than the .jpg too.

I thought I better leave the merged item as a .tif, since converting it
back to a .jpg (or .gif) might alter the result .. and posting it as a
.bmp would have been horrible.

This is mighty confusing. What I think you mean is that you have
scanned your map at three different resolutions 150, 400 and 800 dpi
you have saved these as GIF and JPEG and then stitched them into a
TIFF. As for GIF production, that's a whole can of worms. How many
colours, which colours, to dither or not to dither?

For what it's worth I reckon:

TL = GIF at 400
TR = JPEG at 400
BL = JPEG at 800
BR = GIF at 150

The scans we are all familiar with on the streetmap and other OS
derived digital maps are at 200 dpkm which is actually 100 dpcm which
translates as 250 dpi. There isn't much point in scanning above this
dpi so go back and scan your map at no more than 300 dpi.
--
Phil Cook looking north over the park to the "Westminster Gasworks"

Bitstring <[email hidden]>, from the
wonderful person Phil Cook <[email hidden]> said

Quoted message said:

GSV Three Minds in a Can said:

(polling Paul, if he's out there) .. After the discussions of .jpg vs
.gif (or .bmp or .TIF or whatever) for compressing and saving maps, I've
set up a blind test sample at:

www.quik.clara.co.uk/samples.tif

, which contains a 'stitch up' of two .gifs (150 bpi and 400 bpi - 1.7Mb
and 9.9 Mb respectively) and two .jpgs (400 bpi and 800 bpi - 3.9 and
17.3Mb). The sizes are for a 'full scanner page' of map (1947 OS
sheet129, as it happens).

Your challenge, should you choose to accept it (it's a 1.7Mb download,
as a .tif file) is to decide which of the 4 segments was which, and why
the .gif beats the .jog, give that (at 400 bpi) it's twice as big. or
maybe I used the wrong .gif production program?

I can actually see more artefacts in the .gif (random lighter speckles)
than the .jpg too.

I thought I better leave the merged item as a .tif, since converting it
back to a .jpg (or .gif) might alter the result .. and posting it as a
.bmp would have been horrible.

This is mighty confusing. What I think you mean is that you have
scanned your map at three different resolutions 150, 400 and 800 dpi
you have saved these as GIF and JPEG and then stitched them into a
TIFF.

Correct! Well, actually I stitched them in .pdd form, and output the
result as a TIF(f).

Quoted message said:

As for GIF production, that's a whole can of worms. How many
colours, which colours, to dither or not to dither?

255 colours. Whatever the default for photodeluxe Gif98a output is.

Quoted message said:

For what it's worth I reckon:

TL = GIF at 400
TR = JPEG at 400
BL = JPEG at 800
BR = GIF at 150

You have the TL and TR reversed, but the other two are right. Well, the
150 bpi GIF is a bit easy, coz the result is pretty lousy.

Quoted message said:

The scans we are all familiar with on the streetmap and other OS
derived digital maps are at 200 dpkm which is actually 100 dpcm which
translates as 250 dpi. There isn't much point in scanning above this
dpi so go back and scan your map at no more than 300 dpi.

I'm iterating towards it .. 150 dpi certainly seemed like too little,
and 800 dpi was too much, so I guess the answer will be in the 200-300
ballpark. However unless there is a better .gif output option, I think I
shall go with .jpgs and to heck with the 'you'll get artefacts and you
don't need 24 bit colour anyway' brigade. 8>.

--
GSV Three Minds in a Can
Outgoing Msgs are Turing Tested,and indistinguishable from human typing.

I find it's worth doing a sharpen filter after scanning as this really
improves the quality. I have some 1950's 7th series OS maps that I've
scanned and sharpened and they look nearly as good as the modern
Streetmap 1:50,000 images. Personally I use 24bit PNG so as not to loose
any quality, the downside is obviously the size but I'm working on that
by breaking the very large images into tiles (like Streetmap) and just
loading the ones that are visible.

--
Andrew Whaley, author of :-

Trailgauge - Shareware 3D GPS Mapping Software
Free Download from http://www.trailgauge.com

Bitstring <[email hidden]>, from the wonderful person
Andrew Whaley <[email hidden]> said

Quoted message said:

I find it's worth doing a sharpen filter after scanning as this really
improves the quality. I have some 1950's 7th series OS maps that I've
scanned and sharpened and they look nearly as good as the modern
Streetmap 1:50,000 images. Personally I use 24bit PNG so as not to
loose any quality, the downside is obviously the size but I'm working
on that by breaking the very large images into tiles (like Streetmap)
and just loading the ones that are visible.

Thanks Andrew, I'll give that a try. I was considering loading the final
scans on a website so I think PNG is out of the question. Since I only
want them for oziexplorer 'tracing' of features like rivers, roads, etc.
(and if I get really really bored, contours) absolute quality is not an
issue, although it's nice to be able to read the names and see adequate
detail to trace.

For 'my own use' I've got 2001 OS data (digital) and Tracklogs, and I
can output digital tiles (using screen copy/paste) into Oziexplorer,
which gets round all the scanning/distortion issues. However that is
copyright and I can't post it on the web, nor post Garmin .img files
derived from it.

--
GSV Three Minds in a Can
Outgoing Msgs are Turing Tested,and indistinguishable from human typing.

GSV Three Minds in a Can said:

Bitstring <[email hidden]>, from the
wonderful person Phil Cook <[email hidden]> said

Quoted message said:

GSV Three Minds in a Can said:

(polling Paul, if he's out there) .. After the discussions of .jpg vs
.gif (or .bmp or .TIF or whatever) for compressing and saving maps, I've
set up a blind test sample at:

www.quik.clara.co.uk/samples.tif

, which contains a 'stitch up' of two .gifs (150 bpi and 400 bpi - 1.7Mb
and 9.9 Mb respectively) and two .jpgs (400 bpi and 800 bpi - 3.9 and
17.3Mb). The sizes are for a 'full scanner page' of map (1947 OS
sheet129, as it happens).

Your challenge, should you choose to accept it (it's a 1.7Mb download,
as a .tif file) is to decide which of the 4 segments was which, and why
the .gif beats the .jog, give that (at 400 bpi) it's twice as big. or
maybe I used the wrong .gif production program?

I can actually see more artefacts in the .gif (random lighter speckles)
than the .jpg too.

I thought I better leave the merged item as a .tif, since converting it
back to a .jpg (or .gif) might alter the result .. and posting it as a
.bmp would have been horrible.

This is mighty confusing. What I think you mean is that you have
scanned your map at three different resolutions 150, 400 and 800 dpi
you have saved these as GIF and JPEG and then stitched them into a
TIFF.

Correct! Well, actually I stitched them in .pdd form, and output the
result as a TIF(f).

Quoted message said:

As for GIF production, that's a whole can of worms. How many
colours, which colours, to dither or not to dither?

255 colours. Whatever the default for photodeluxe Gif98a output is.

Photodeluxe... is that the pre-elements Adobe s/w? I can remember
running that on a 486 with 12Meg way back... I soon ditched it when I
got PS LE 4.0 bundled with a scanner.

Quoted message said:

Quoted message said:

For what it's worth I reckon:

TL = GIF at 400
TR = JPEG at 400
BL = JPEG at 800
BR = GIF at 150

You have the TL and TR reversed, but the other two are right. Well, the
150 bpi GIF is a bit easy, coz the result is pretty lousy.

Interesting, I could have sworn that TL was showing "colour reduction"
artifacts.

I thought of something after I posted my reply that in order to get
rid of the specking in light areas you need to adjust the levels in
the scan. An automatic scan will try to reproduce the map darker than
it is, so to get the white back to white force it to clip in your
picture editing s/w.

I've just done this with your samples.tif and converted it to a
uniform 256 colour GIF

http://www.p-t-cook.freeserve.co.uk/temp/samples.gif

Quoted message said:

Quoted message said:

The scans we are all familiar with on the streetmap and other OS
derived digital maps are at 200 dpkm which is actually 100 dpcm which
translates as 250 dpi. There isn't much point in scanning above this
dpi so go back and scan your map at no more than 300 dpi.

I'm iterating towards it .. 150 dpi certainly seemed like too little,
and 800 dpi was too much, so I guess the answer will be in the 200-300
ballpark. However unless there is a better .gif output option, I think I
shall go with .jpgs and to heck with the 'you'll get artefacts and you
don't need 24 bit colour anyway' brigade. 8>.

You might want to look at how your mapping s/w handles image files.
Most types of file will need to be fully loaded but some can be paged,
just the bit that is needed is loaded. This makes for quicker loading
of maps, though it might not be a problem if your computer is fairly
recent and has blistering speed.
--
Phil Cook looking north over the park to the "Westminster Gasworks"

Bitstring <[email hidden]>, from the
wonderful person Phil Cook <[email hidden]> said
<sbig snip>

Quoted message said:

I've just done this with your samples.tif and converted it to a
uniform 256 colour GIF

What did you use to produce the .gif? My 'photodeluxe' (adobe) seems to
only want to produce rather lousy .gifs (I think it dithers, whether you
want it or not).

Quoted message said:

Quoted message said:

Quoted message said:

The scans we are all familiar with on the streetmap and other OS
derived digital maps are at 200 dpkm which is actually 100 dpcm which
translates as 250 dpi. There isn't much point in scanning above this
dpi so go back and scan your map at no more than 300 dpi.

I'm iterating towards it .. 150 dpi certainly seemed like too little,
and 800 dpi was too much, so I guess the answer will be in the 200-300
ballpark. However unless there is a better .gif output option, I think I
shall go with .jpgs and to heck with the 'you'll get artefacts and you
don't need 24 bit colour anyway' brigade. 8>.

You might want to look at how your mapping s/w handles image files.
Most types of file will need to be fully loaded but some can be paged,
just the bit that is needed is loaded. This makes for quicker loading
of maps, though it might not be a problem if your computer is fairly
recent and has blistering speed.

One of my other amusements is building PCs, so speed isn't a big problem
(although manipulating 28" square .jpgs at 200 dpi still strains things
a bit - I may have to throw in some more RAM).

Rights now I'm going quietly mad trying to get the scanned pieces sewn
back together (12 or 16 of them) without obvious joins (and with the
grid lines more of less where they should be). The map has had 50+ years
for the creases to set, so getting it flat enough for an undistorted
scan (and manipulating it on the scanner) is a real pain.

I can get reasonable assembly as far as 4 quadrants, but putting them
together into a whole 1" OS sheet is proving rather tougher than I
expected. 8<,

--
GSV Three Minds in a Can
Outgoing Msgs are Turing Tested,and indistinguishable from human typing.

GSV Three Minds in a Can said:

Bitstring <[email hidden]>, from the
wonderful person Phil Cook <[email hidden]> said
<sbig snip>

Quoted message said:

I've just done this with your samples.tif and converted it to a
uniform 256 colour GIF

What did you use to produce the .gif? My 'photodeluxe' (adobe) seems to
only want to produce rather lousy .gifs (I think it dithers, whether you
want it or not).

I used Photoshop. The important bit was messing with the levels to
make the white of the paper more white. I somehow doubt whether
Photodeluxe has a levels command :-( That gif was made using the bog
standard settings in PS with a uniform set of 256 colours. Being more
selective with the colours can reduce the file size a bit more. I
think streetmap use GIFs with a selection of the overall 256 colours
used in the map as a whole. The less colours in each tile the smaller
the GIF, a plain sea tile (200x200 pix) comes out at about 7k whilst
an urban one with lots of different colours can be in the region of
32k.

Quoted message said:

Rights now I'm going quietly mad trying to get the scanned pieces sewn
back together (12 or 16 of them) without obvious joins (and with the
grid lines more of less where they should be). The map has had 50+ years
for the creases to set, so getting it flat enough for an undistorted
scan (and manipulating it on the scanner) is a real pain.

I can get reasonable assembly as far as 4 quadrants, but putting them
together into a whole 1" OS sheet is proving rather tougher than I
expected. 8<,

Now you know why people nick tiles off streetmap :-)

If you are going to get serious about this map scanning I think you
need some better image software than Photodeluxe and or some image
stitching software, or a mate with an A0 scanner.
--
Phil Cook looking north over the park to the "Westminster Gasworks"

GSV Three Minds in a Can said:

Rights now I'm going quietly mad trying to get the scanned pieces sewn
back together (12 or 16 of them) without obvious joins (and with the
grid lines more of less where they should be). The map has had 50+ years
for the creases to set, so getting it flat enough for an undistorted
scan (and manipulating it on the scanner) is a real pain.

I can get reasonable assembly as far as 4 quadrants, but putting them
together into a whole 1" OS sheet is proving rather tougher than I
expected. 8<,

You could use Ozi Mapmerge to merge lots of small maps (each using an
image from a single scan) but there is a catch, you won't be able to
output the image to a file, you'll have to use screen capture and
assemble them into an image if you want to use it outside of Ozi.

http://216.218.220.254/mapmerge/mapmerge.html
--
Phil Cook looking north over the park to the "Westminster Gasworks"

Bitstring <[email hidden]>, from the
wonderful person [email hidden] said

Quoted message said:

GSV Three Minds in a Can said:

Rights now I'm going quietly mad trying to get the scanned pieces sewn
back together (12 or 16 of them) without obvious joins (and with the
grid lines more of less where they should be). The map has had 50+ years
for the creases to set, so getting it flat enough for an undistorted
scan (and manipulating it on the scanner) is a real pain.

I can get reasonable assembly as far as 4 quadrants, but putting them
together into a whole 1" OS sheet is proving rather tougher than I
expected. 8<,

Look back to the earlier suggestion to use a (free) mapping package.

Suggestions (or do you mean Oziexplorer?)? I looked for 'earlier
suggestion' but didn't identify any package name.

Quoted message said:

Each scan can then be ortho rectified, and calibrated, and placed as a
tile, all tiles then being loaded to view the whole picture. This can
then be output to a large printer format.

I actually don't need it printed; I just need a 'whole sheet' .gif or
..jpg that I can post on the www (so size is an issue) and things like
Oziexplorer and MapEdit.

--
GSV Three Minds in a Can
Outgoing Msgs are Turing Tested,and indistinguishable from human typing.

I'm a great fan of TIFF, it is the industry standard, TIFF has various
formats, one of which is LZW compression, this will reduce the file size but
not loose any data, you can then use your favourite graphics software to
convert to JPG or whatever you want.

An excellent freeware package to convert file formats is XnView frrom
http://xnview.com/ that will tell you all you ever whant to know about what
the TIFF format is

--

Ted Ferenc. (http://www.ndrw.co.uk)
This message, and any attachment, is private and confidential.
It is intended only for the named addressee(s).
"GSV Three Minds in a Can" <[email hidden]> wrote in message
news:[email hidden]...

Quoted message said:

(polling Paul, if he's out there) .. After the discussions of .jpg vs
.gif (or .bmp or .TIF or whatever) for compressing and saving maps, I've
set up a blind test sample at:

www.quik.clara.co.uk/samples.tif

, which contains a 'stitch up' of two .gifs (150 bpi and 400 bpi - 1.7Mb
and 9.9 Mb respectively) and two .jpgs (400 bpi and 800 bpi - 3.9 and
17.3Mb). The sizes are for a 'full scanner page' of map (1947 OS
sheet129, as it happens).

Your challenge, should you choose to accept it (it's a 1.7Mb download,
as a .tif file) is to decide which of the 4 segments was which, and why
the .gif beats the .jog, give that (at 400 bpi) it's twice as big. or
maybe I used the wrong .gif production program?

I can actually see more artefacts in the .gif (random lighter speckles)
than the .jpg too.

I thought I better leave the merged item as a .tif, since converting it
back to a .jpg (or .gif) might alter the result .. and posting it as a
.bmp would have been horrible.

--
GSV Three Minds in a Can
Outgoing Msgs are Turing Tested,and indistinguishable from human typing.

Ted Ferenc said:

I'm a great fan of TIFF, it is the industry standard, TIFF has various
formats, one of which is LZW compression, this will reduce the file size but
not loose any data, you can then use your favourite graphics software to
convert to JPG or whatever you want.

The LZW compression algorithm costs the software writer a licence fee
so not all programs can handle it. Also you can't open a TIFF in your
browser which I think is what the OP wants.

Quoted message said:

An excellent freeware package to convert file formats is XnView frrom
http://xnview.com/ that will tell you all you ever whant to know about what
the TIFF format is

Great for converting files from one format to another but next to
useless for editing.

As for freeware editing s/w I have just discovered PhotoFiltre
http://photofiltre.free.fr/frames_en.htm
--
Phil Cook looking north over the park to the "Westminster Gasworks"

"Phil Cook" <[email hidden]> wrote in message
news:[email hidden]...

Quoted message said:

Ted Ferenc said:

I'm a great fan of TIFF, it is the industry standard, TIFF has various
formats, one of which is LZW compression, this will reduce the file size

but

Quoted message said:

Quoted message said:

not loose any data, you can then use your favourite graphics software to
convert to JPG or whatever you want.

The LZW compression algorithm costs the software writer a licence fee
so not all programs can handle it. Also you can't open a TIFF in your
browser which I think is what the OP wants.

Yes I know, but there is also packbits which is not copyrighted, which in
some circumstances is better than LZW. My point is that it is better to scan
use TIFF or any format which is not lossy then you don't loose quality when
you manipulate the imags or saving as JPG

Quoted message said:

Quoted message said:

An excellent freeware package to convert file formats is XnView frrom
http://xnview.com/ that will tell you all you ever whant to know about

what

Quoted message said:

Quoted message said:

the TIFF format is

Great for converting files from one format to another but next to
useless for editing.

That is what I said

Quoted message said:

As for freeware editing s/w I have just discovered PhotoFiltre
http://photofiltre.free.fr/frames_en.htm
--
Phil Cook looking north over the park to the "Westminster Gasworks"

Ted Ferenc. (http://www.ndrw.co.uk)

In article <[email hidden]>, Phil Cook
<[email hidden]> writes

Quoted message said:

Ted Ferenc said:

I'm a great fan of TIFF, it is the industry standard, TIFF has various
formats, one of which is LZW compression, this will reduce the file size but
not loose any data, you can then use your favourite graphics software to
convert to JPG or whatever you want.

The LZW compression algorithm costs the software writer a licence fee
so not all programs can handle it.

Not anymore, the patent has expired.

http://www.unisys.com/about__unisys/lzw

However there is still a considerable legacy of programs that do not
have LZW capability and likely will never have it due to Unisys.

--

Dominic Sexton
http://www.dscs.demon.co.uk/

In article <[email hidden]>, Phil Cook
<[email hidden]> writes

Quoted message said:

Ted Ferenc said:

I'm a great fan of TIFF, it is the industry standard, TIFF has various
formats, one of which is LZW compression, this will reduce the file size but
not loose any data, you can then use your favourite graphics software to
convert to JPG or whatever you want.

The LZW compression algorithm costs the software writer a licence fee
so not all programs can handle it.

Not anymore, the patent has expired.

http://www.unisys.com/about__unisys/lzw

However there is still a considerable legacy of programs that do not
have LZW capability and likely will never have it due to Unisys.

--

Dominic Sexton
http://www.dscs.demon.co.uk/

Bitstring <[email hidden]>, from the
wonderful person [email hidden] said

Quoted message said:

GSV Three Minds in a Can said:

Suggestions (or do you mean Oziexplorer?)? I looked for 'earlier
suggestion' but didn't identify any package name.

I've never tried or seen oziexplorer, I imagine a number of the
mapping packages will do it as well as GIS ones.

Quoted message said:

Quoted message said:

Each scan can then be ortho rectified, and calibrated, and placed as a
tile, all tiles then being loaded to view the whole picture. This can
then be output to a large printer format.

I actually don't need it printed; I just need a 'whole sheet' .gif or
.jpg that I can post on the www (so size is an issue) and things like
Oziexplorer and MapEdit.

Well for posting on the web in simple html you cannot use bigger than
1200 by 800 pixels can you?

Not for direct viewing, but people can 'download' a file from a website
of arbitrary size. However having thought about it some more, I doubt a
~20Mbyte GIF of .JPG is going to be either popular, or much use to the
recipient (Photodeluxe struggles to do anything with a file that big,
and
PhotFiltre just curls up and dies with a 'not enough storage to complete
the operation' message). I wondered why I couldn't find any 'out of
copyright OS maps' on the www, and now I think I see why .. needs a
seriously large amount of space. Sticking them on CD may be saner.

Anyway, it's probably better if I just post the A4 tiles (maybe having
trimmed the distorted edges a bit), along with the calibration data ..
anyone who wants to stick them together can always do so.

OziExplorer's 'map merge utility' actually does a pretty good job, but
sadly produces only a proprietary format, which you can use in the
program but can't use elsewhere. Then again, 'Mapedit' (which is my
primary need outside OziExplorer) chokes on anything over about 5Mb,
since it seems to be trying to use the graphics card &/or DirectX to do
some of the image handling.

Quoted message said:

I thought you wanted to reprint the sheet. The gps trackmaker will do
what you want and then capture using printscreen.

With a map scan I would aim to calibrate on 4 grid intersections for
each tile, load the tiles and then either save the screen image at the
pixel width count for all the tiles in view (potentially a huge file)
or capture the screen image to clipboard.

I think it should be possible to create a file output for a printer
and then convert this back to a big image. My software allows me to
emulate an A0 printer and produces A4 tiles which I can paste
together.

If you want to let me have a try you could let me have 4 scans with
full grid references for the top left and bottom right grid
intersections of each tile and I will try them in gps track and map
maker.

Thanks for the offer, but I'll play around some more with 'pieces' for a
while. I guess I do need to go off and update my image manipulation
software though - Photodeluxe 4.0, and Photoshop LE 5.0 appear to be
well past their 'best before' data, and my copy of PhotoImpact is even
older.. 8<,

I assume Adobe/Photshop is still the best game in town, and that the
full blown version is still outrageously expensive???

--
GSV Three Minds in a Can
Outgoing Msgs are Turing Tested,and indistinguishable from human typing.

GSV Three Minds in a Can said:

PhotFiltre just curls up and dies with a 'not enough storage to complete
the operation' message).

Interesting. I haven't tried it on big stuff but it seemed pretty good
for a freebie.

Quoted message said:

I guess I do need to go off and update my image manipulation
software though - Photodeluxe 4.0, and Photoshop LE 5.0 appear to be
well past their 'best before' data, and my copy of PhotoImpact is even
older.. 8<,

I assume Adobe/Photshop is still the best game in town, and that the
full blown version is still outrageously expensive???

Yes but PS Elements3, which is the descendant of PS LE 5.0, is more
affordable.
--
Phil Cook looking north over the park to the "Westminster Gasworks"

Dominic Sexton said:

In article <[email hidden]>, Phil Cook
<[email hidden]> writes

Quoted message said:

Ted Ferenc said:

I'm a great fan of TIFF, it is the industry standard, TIFF has various
formats, one of which is LZW compression, this will reduce the file size but
not loose any data, you can then use your favourite graphics software to
convert to JPG or whatever you want.

The LZW compression algorithm costs the software writer a licence fee
so not all programs can handle it.

Not anymore, the patent has expired.

http://www.unisys.com/about__unisys/lzw

However there is still a considerable legacy of programs that do not
have LZW capability and likely will never have it due to Unisys.

Hmm.How long does a patent last? ...20 years. So LZW is what you call
mature technology. The trouble is somebody can tinker with it and
improve enough and gets a new patent. :-(
--
Phil Cook looking north over the park to the "Westminster Gasworks"