[OSM-talk] Heresy - pure discussion

[Alexandre Oliveira]

I think it's stupid to even think about switching to MSSQL. OSM is
free and open data built on top of free and open source technologies.
Switching to MSSQL would be a dumb move, and as mentioned before, it
would require the refactor of several tools built to work with
PostgreSQL, which is pretty much the standard database nowadays for
geospatial data, and IMO it would make OSM non free and open.

> Having said that the main advantage of SQL is
> it is a standard so you should be able to connect practically anything to
> it.

That's not entirely true. SQL is a language but every database
implements its own dialect, i.e., some query keywords implemented in
MSSQL might not be available in MySQL/MariaDB and vice-versa.


> For development Microsoft visual studio is normally recognised as the gold
> standard for development environments and remember Github is now owned by
> Microsoft.

Gold standard for Microsoft owned products and languages. Would you
use Visual Studio to develop a Python app? Of course not, there are
much better IDEs out there. Visual Studio is widely used for Microsoft
stuff only - C#, F#, Microsoft SQL Server, and others.


I think switching to MSSQL is dumb. For the reasons mentioned above,
and what would we get from it? Microsoft support and an increase in
expenses by both users and OSM "employees" because we now need to pay
Microsoft to use their stuff? Is it really worth the hassle? Besides,
there's the side effect that many tech-savvy people would just abandon
OSM if they had to adapt their stuff to Microsoft's standards.


On 7/24/20, john whelan <jwhelan0112 at gmail.com> wrote:
> You need to define the requirements and if having open source software is a
> top priority that's fine.
>
> If reliability and security are critical then you have to start balancing
> things out.
>
> In general UNIX based solutions do not have the same tools available in
> Windows but with a skilled administrator they can be made reasonably
> reliable and secure but a higher level of skill is required in the UNIX
> environment.  Skilled administrator time from volunteers is not expensive
> having said that we should be asking them to do more work than we could?
>
> The 300 users and 600 accounts was actually on a Microsoft SQL server where
> each database user was given their own account and password.   No record of
> who was given which account was kept and over the years people came and
> left.  I agree the admins were at fault but over the years there had been a
> number of admins some had more expertise than others and to an outsider
> knowing which knew what they were doing and which were basically bluffing
> is not always easy.  We had probably fifty database administrators besides
> my team all doing their own thing.  On the Microsoft SQL databases where we
> used Windows operating system groups if someone left they were removed from
> the group and we could check with their admin if they were part of the
> section or not.
>
> Microsoft SQL Server Express is a free limited version of SQL server that
> may well do for many users.  Having said that the main advantage of SQL is
> it is a standard so you should be able to connect practically anything to
> it.
>
> For development Microsoft visual studio is normally recognised as the gold
> standard for development environments and remember Github is now owned by
> Microsoft.
>
> Cheerio John
>
> On Fri, Jul 24, 2020, 17:03 Yves <ycai at mailbox.org> wrote:
>
>> But face it, philosophy is now also part of the discussion. And that's
>> important.
>> Yves
>>
>> Le 24 juillet 2020 20:50:22 GMT+02:00, john whelan
>> <jwhelan0112 at gmail.com>
>> a écrit :
>>>
>>> If the database was smaller and less infrastructure was reliant on it
>>> working I would agree with you that philosophically open source software
>>> makes a lot of sense.
>>>
>>> However your argument is philosophical rather than logical.
>>>
>>> Note I'm merely requesting that the idea be examined.  I am not saying I
>>> know what is best and all the things that need to be considered.
>>>
>>> Cheerio John
>>>
>>> On Fri, Jul 24, 2020, 14:35 Yves <ycai at mailbox.org> wrote:
>>>
>>>> You're probably have some very good points when it comes to database
>>>> management, but running an open map on open source software makes a lot
>>>> of
>>>> sense.
>>>>
>>>> Yves
>>>>
>>>> Le 24 juillet 2020 20:11:46 GMT+02:00, john whelan <
>>>> jwhelan0112 at gmail.com> a écrit :
>>>>>
>>>>> All this talk about databases and servers and sysadmins makes me
>>>>> wonder
>>>>> if we should reconsider our choice of operating systems and databases.
>>>>>
>>>>> At one time in the past I ran a Database support group that covered
>>>>> Sybase, Oracle, Microsoft SQL server, ingres and half a dozen other
>>>>> database systems.
>>>>>
>>>>> The UNIX side, some twenty or so servers ran software that in theory
>>>>> monitored the databases.  In practise it never really was upto date.
>>>>> Microsoft also had a very nice monitoring tool that monitored and
>>>>> suggested
>>>>> solutions.  I've dropped an example report below.
>>>>>
>>>>> We ran probably fifty SQL server database servers and I spent quite a
>>>>> lot of time maxing the memory on a server then consolidating servers.
>>>>> Towards the end we had far more data running on SQL server than we did
>>>>> on
>>>>> the UNIX side.  The servers were cheaper for the same performance for
>>>>> a
>>>>> start.
>>>>>
>>>>> Many of the UNIX based servers had default passwords set which made
>>>>> security a problem.  Fortunately they were protected by an air gap from
>>>>> the
>>>>> Internet.
>>>>>
>>>>> We had an IBM mainframe in the mix with an old database on it.  The
>>>>> programmers gradually retired.  I was lucky and identified another
>>>>> government department that was switching away from it and we managed
>>>>> to
>>>>> grab a handful of programmers etc from them.  Then a couple of years
>>>>> later
>>>>> that DBA retired.  You need to think of the future.  Will I be able to
>>>>> get
>>>>> knowledgeable staff if I need to?  We had to pay the company to run a
>>>>> special course in Ottawa and that was not cheap by the time we put the
>>>>> trainer up in a hotel and paid his airfare from the states.
>>>>>
>>>>> Initially the Microsoft side suffered from lack of security but they
>>>>> hardened the operating system and SQL server to a point where it was
>>>>> the
>>>>> most secure combination.  Microsoft SQL server was originally Sybase
>>>>> but
>>>>> got completely rewritten over time.
>>>>>
>>>>> On the support side my staff found that once we had set the
>>>>> permissions
>>>>> to an operating system group we just had to add people to the group.
>>>>> For
>>>>> other databases each person had to be given permissions individually
>>>>> which
>>>>> made for finger problems.  The classic was one secure database that
>>>>> was
>>>>> supposed to be accessed operationally by 300 people. The problem was
>>>>> there
>>>>> were 600 accounts and no one knew which ones were needed or which could
>>>>> be
>>>>> deleted to reduce the surface area for attack.
>>>>>
>>>>> The integrated Microsoft monitoring system made reliability much
>>>>> better.  There were far fewer problems on the Microsoft SQL side than
>>>>> on
>>>>> the UNIX / other database side and they were easier to fix.  One of my
>>>>> less
>>>>> expert database admins was shocked by the ease of which he caught the
>>>>> problem and corrected it by himself after an alert.  It gave him a bit
>>>>> of
>>>>> confidence as well.
>>>>>
>>>>> We changed to PostgreSQL in 2009.  The size of the database was much
>>>>> smaller then.
>>>>>
>>>>> One thing we noticed was on the database tuning side.  SQL server
>>>>> worked better if you just left it alone and didn't try to tune it.  It
>>>>> would check what was in memory rather than go out to the disk drives
>>>>> and
>>>>> that made a big difference to performance.  We measure disk access in
>>>>> milliseconds and memory access in nanoseconds.  One is ten thousand
>>>>> times
>>>>> smaller than the other.
>>>>>
>>>>> On the reliability side there is a set of guidelines that are
>>>>> basically
>>>>> common sense.  I forget the formal (ISO?) name but many organisations
>>>>> have
>>>>> seen considerable savings in money and in reliability by using them.  I
>>>>> met
>>>>> the English guy who originated them at a Microsoft presentation.  They
>>>>> can
>>>>> be applied to any environment.
>>>>>
>>>>> I think we either run the largest PostgreSQL database there is or it
>>>>> is close to it.  From a reliability point of view my professional hat
>>>>> says
>>>>> this is not where you want to be. You want to be more mainstream with
>>>>> someone else being on the bleeding edge.
>>>>>
>>>>> So the heresy would be look at the implications of changing to
>>>>> Microsoft SQL server in the cloud.  There is lots of documentation and
>>>>> given that Microsoft has worked closely with us in the past the cost
>>>>> might
>>>>> not be too bad.  I do understand that we have a large investment in
>>>>> our
>>>>> current set up both as an organisation and personally and many will
>>>>> consider this as heresy but now is probably the time to think about
>>>>> it.
>>>>>
>>>>> Cheerio John
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Your message to Rolland.desrocher at motioncares.ca couldn't be
>>>>> delivered.
>>>>> Rolland.desrocher wasn't found at motioncares.ca.
>>>>> jwhelan0112 Office 365 Rolland.desrocher
>>>>> *Action Required*
>>>>> Recipient
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Unknown To address
>>>>>
>>>>>
>>>>> How to Fix It
>>>>> The address may be misspelled or may not exist. Try one or more of the
>>>>> following:
>>>>>
>>>>>    - Send the message again following these steps: In Outlook, open
>>>>>    this non-delivery report (NDR) and choose *Send Again* from the
>>>>>    Report ribbon. In Outlook on the web, select this NDR, then select
>>>>> the link
>>>>>    "*To send this message again, click here.*" Then delete and retype
>>>>>    the entire recipient address. If prompted with an Auto-Complete
>>>>> List
>>>>>    suggestion don't select it. After typing the complete address,
>>>>> click
>>>>>    *Send*.
>>>>>    - Contact the recipient (by phone, for example) to check that the
>>>>>    address exists and is correct.
>>>>>    - The recipient may have set up email forwarding to an incorrect
>>>>>    address. Ask them to check that any forwarding they've set up is
>>>>> working
>>>>>    correctly.
>>>>>    - Clear the recipient Auto-Complete List in Outlook or Outlook on
>>>>>    the web by following the steps in this article: Fix email delivery
>>>>>    issues for error code 5.1.10 in Office 365
>>>>>    <https://go.microsoft.com/fwlink/?LinkId=532972>, and then send the
>>>>>    message again. Retype the entire recipient address before selecting
>>>>>    *Send*.
>>>>>
>>>>> If the problem continues, forward this message to your email admin. If
>>>>> you're an email admin, refer to the *More Info for Email Admins*
>>>>> section below.
>>>>>
>>>>
>>>> --
>>>> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser
>>>> ma brièveté.
>>>>
>>>
>> --
>> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser
>> ma
>> brièveté.
>>
>


-- 
Atenciosamente,
Alexandre Oliveira.