Mercurial > hg > hg-fastimport

annotate hgext3rd/fastimport/hgimport.py @ 95:3b398a887b95

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Use a sqlite3 database to store the blob data if available This is much more performant than using a filesystem when we are dealing with a large number of blobs. If sqlite3 is not available, then fallback to writing to the filesystem. In both cases, the blob data is compressed before writing to save space. A new option has also been added to specify a path for persistent blob data. This is only really important for large continuous interations where the source data has no concept of export marks and thus only gets bigger. What we gain here is a reduction in the write load on the disk.
author Roy Marples <roy@marples.name>
date Thu, 21 Jan 2021 23:59:21 +0000
parents 2ce33511de87
children cde0e1d24e58
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
1 # Copyright (C) 2008 Canonical Ltd
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
2 #
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
3 # This program is free software; you can redistribute it and/or modify
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
4 # it under the terms of the GNU General Public License as published by
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
5 # the Free Software Foundation; either version 2 of the License, or
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
6 # (at your option) any later version.
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
7 #
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
8 # This program is distributed in the hope that it will be useful,
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
9 # but WITHOUT ANY WARRANTY; without even the implied warranty of
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
10 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
11 # GNU General Public License for more details.
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
12 #
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
13 # You should have received a copy of the GNU General Public License
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
14 # along with this program; if not, write to the Free Software
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
15 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
16
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
17 """Processor of import commands.
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
18
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
19 This module provides core processing functionality including an abstract class
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
20 for basing real processors on. See the processors package for examples.
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
21 """
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
22
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
23 import os
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
24 import shutil
62
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
25 import stat
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
26 import sys
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
27
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
28 from contextlib import closing
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
29
55
a88f0dd05e92 Adapt to Mercurial 1.6 API change.
Greg Ward <greg-hg@gerg.ca>
parents: 52
diff changeset
30 from hgext.convert import common, hg as converthg
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
31 from mercurial import util, zstd
73
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
32 from mercurial.i18n import _
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
33
89
e6602cc471d5 Use vendor/python_fastimport rather than depending on the library
Roy Marples <roy@marples.name>
parents: 79
diff changeset
34 from .vendor.python_fastimport import processor, parser
55
a88f0dd05e92 Adapt to Mercurial 1.6 API change.
Greg Ward <greg-hg@gerg.ca>
parents: 52
diff changeset
35
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
36 # When dealing with many blobs, a database is more efficient
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
37 # than a filesystem. However, it's not necessary for us to work.
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
38 try:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
39 import sqlite3
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
40 have_sqlite = True
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
41 except (ImportError):
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
42 have_sqlite = False
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
43
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
44
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
45 class fastimport_source(common.converter_source):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
46 """Interface between the fastimport processor below and Mercurial's
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
47 normal conversion infrastructure.
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
48 """
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
49 def __init__(self, ui, repotype, repo, sources, blobpath):
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
50 self.ui = ui
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
51 self.sources = sources
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
52 self.processor = HgImportProcessor(ui, repo, blobpath)
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
53 self.parsed = False
73
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
54 self.repotype = repotype
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
55
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
56 # converter_source methods
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
57
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
58 def before(self):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
59 self.processor.setup()
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
60
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
61 def after(self):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
62 self.processor.teardown()
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
63
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
64 def getheads(self):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
65 """Return a list of this repository's heads"""
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
66 self._parse()
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
67 allheads = []
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
68 for branchheads in self.processor.branchmap.values():
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
69 allheads.extend(branchheads)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
70 return allheads
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
71
68
47bc97e1c46f Drop support for Mercurial <= 1.5.
Greg Ward <greg@gerg.ca>
parents: 66
diff changeset
72 def getfile(self, name, fileid):
47bc97e1c46f Drop support for Mercurial <= 1.5.
Greg Ward <greg@gerg.ca>
parents: 66
diff changeset
73 if fileid is None: # deleted file
73
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
74 return None, None
68
47bc97e1c46f Drop support for Mercurial <= 1.5.
Greg Ward <greg@gerg.ca>
parents: 66
diff changeset
75 return (self.processor.getblob(fileid),
47bc97e1c46f Drop support for Mercurial <= 1.5.
Greg Ward <greg@gerg.ca>
parents: 66
diff changeset
76 self.processor.getmode(name, fileid))
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
77
73
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
78 def getchanges(self, commitid, full):
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
79 """Returns a tuple of (files, copies, cleanp2).
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
80
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
81 files is a sorted list of (filename, id) tuples for all files
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
82 changed between commitid and its first parent returned by
73
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
83 getcommit().
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
84 commitid id is the source revision id of the file.
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
85 cleanp2 is currently unused and an empty set is returned.
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
86
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
87 copies is a dictionary of dest: source
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
88 """
73
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
89 if full:
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
90 raise util.Abort(_("convert from fastimport does not support --full"))
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
91 return (self.processor.modified[commitid],
73
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
92 self.processor.copies[commitid],
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
93 set())
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
94
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
95 def getcommit(self, commitid):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
96 """Return the commit object for commitid"""
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
97 if commitid is None:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
98 return None
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
99 else:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
100 return self.processor.commitmap[commitid]
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
101
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
102 def gettags(self):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
103 """Return the tags as a dictionary of name: revision"""
49
ed66bd7bd2f6 comment
Greg Ward <greg-hg@gerg.ca>
parents: 48
diff changeset
104 # oops, this loses order
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
105 return dict(self.processor.tags)
92
efe678b023d3 Whitespace police
Roy Marples <roy@marples.name>
parents: 89
diff changeset
106
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
107 def getchangedfiles(self, rev, i):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
108 """Return the files changed by rev compared to parent[i].
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
109
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
110 i is an index selecting one of the parents of rev. The return
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
111 value should be the list of files that are different in rev and
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
112 this parent.
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
113
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
114 If rev has no parents, i is None.
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
115
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
116 This function is only needed to support --filemap
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
117 """
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
118 raise NotImplementedError()
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
119
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
120 # private worker methods
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
121
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
122 def _parse(self):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
123 if self.parsed:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
124 return
62
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
125 for source in self.sources:
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
126 if source == b"-":
62
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
127 infile = sys.stdin
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
128 else:
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
129 infile = open(source, 'rb')
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
130 try:
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
131 p = parser.ImportParser(infile)
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
132 self.processor.process(p.iter_commands)
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
133 finally:
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
134 if infile is not sys.stdin:
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
135 infile.close()
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
136 self.parsed = True
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
137
62
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
138
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
139 class HgImportProcessor(processor.ImportProcessor):
92
efe678b023d3 Whitespace police
Roy Marples <roy@marples.name>
parents: 89
diff changeset
140
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
141 tagprefix = b"refs/tags/"
58
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
142
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
143 def __init__(self, ui, repo, blobpath):
42
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
144 super(HgImportProcessor, self).__init__()
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
145 self.ui = ui
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
146 self.repo = repo
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
147
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
148 self.commitmap = {} # map commit ID (":1") to commit object
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
149 self.branchmap = {} # map branch name to list of heads
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
150
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
151 # see HgImportCommitHandler for details on these three
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
152 self.modified = {} # map commit id to list of file mods
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
153 self.filemodes = {} # map commit id to {filename: mode} map
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
154 self.copies = {} # map commit id to dict of file copies
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
155
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
156 self.tags = [] # list of (tag, mark) tuples
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
157
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
158 self.numblobs = 0 # for progress reporting
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
159 if blobpath:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
160 self.blobpath = blobpath
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
161 self.blobpersist = True
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
162 else:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
163 self.blobpath = os.path.join(self.repo.root, b".hg", b"blobs")
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
164 self.blobpersist = False
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
165 self.blobdb = None
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
166 self.compressor = zstd.ZstdCompressor(level = 6) # compress the blob
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
167 self.decompressor = zstd.ZstdDecompressor() # decompress the blob
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
168
34
08e2157aaa9a Remove local fork of bzr-fastimport; use my fastimport library instead.
Greg Ward <greg-hg@gerg.ca>
parents: 33
diff changeset
169 def setup(self):
08e2157aaa9a Remove local fork of bzr-fastimport; use my fastimport library instead.
Greg Ward <greg-hg@gerg.ca>
parents: 33
diff changeset
170 """Setup before processing any streams."""
08e2157aaa9a Remove local fork of bzr-fastimport; use my fastimport library instead.
Greg Ward <greg-hg@gerg.ca>
parents: 33
diff changeset
171 pass
08e2157aaa9a Remove local fork of bzr-fastimport; use my fastimport library instead.
Greg Ward <greg-hg@gerg.ca>
parents: 33
diff changeset
172
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
173 def teardown(self):
34
08e2157aaa9a Remove local fork of bzr-fastimport; use my fastimport library instead.
Greg Ward <greg-hg@gerg.ca>
parents: 33
diff changeset
174 """Cleanup after processing all streams."""
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
175 if self.blobdb:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
176 self.blobdb.commit()
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
177 self.blobdb.close()
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
178 self.blobdb = None
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
179 if self.blobpersist:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
180 self.ui.debug(b"Persisting blob database %s ...\n"
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
181 % self.blobpath)
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
182 else:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
183 self.ui.debug(b"Removing blob database %s ...\n"
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
184 % self.blobpath)
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
185 os.remove(self.blobpath)
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
186 elif os.path.exists(self.blobpath):
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
187 if self.blobpersist:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
188 self.ui.debug(b"Persisting blob directory %s ...\n"
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
189 % self.blobpath)
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
190 else:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
191 self.ui.debug(b"Removing blob directory %s ...\n"
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
192 % self.blobpath)
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
193 shutil.rmtree(self.blobpath)
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
194
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
195 def progress_handler(self, cmd):
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
196 self.ui.write(b"Progress: %s\n" % cmd.message)
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
197
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
198 def blob_handler(self, cmd):
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
199 self.writeblob(cmd.id, cmd.data)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
200
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
201 def _openblobdb(self):
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
202 self.blobdb = sqlite3.connect(self.blobpath)
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
203 self.blobdb.text_factory = str
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
204 self.blobdb.execute("CREATE TABLE IF NOT EXISTS blob(id varchar PRIMARY KEY NOT NULL, content blob NOT NULL)")
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
205
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
206 def _getblobfilename(self, blobid):
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
207 # XXX should escape ":" for windows
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
208 return os.path.join(self.blobpath, b"blob-" + blobid)
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
209
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
210 def writeblob(self, blobid, data):
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
211 if self.compressor:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
212 data = self.compressor.compress(data)
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
213
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
214 if have_sqlite:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
215 if self.blobdb is None:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
216 self._openblobdb()
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
217 self.blobdb.execute("INSERT OR IGNORE INTO blob(id, content) VALUES(?, ?)",
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
218 (blobid, data))
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
219 else:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
220 fn = self._getblobfilename(blobid)
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
221 if not os.path.isfile(fn):
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
222 if not os.path.exists(self.blobpath):
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
223 os.mkdir(self.blobpath)
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
224 with open(fn, "wb") as f:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
225 f.write(data)
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
226
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
227 self.numblobs += 1
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
228 if self.numblobs % 500 == 0:
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
229 self.ui.status(b"%d blobs read\n" % self.numblobs)
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
230
95
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
231 def getblob(self, fileid):
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
232 (commitid, blobid) = fileid
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
233
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
234 if have_sqlite:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
235 if self.blobdb is None:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
236 self._openblobdb()
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
237 with closing(self.blobdb.cursor()) as c:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
238 c.execute("SELECT content FROM blob WHERE id=?", (blobid,))
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
239 data, = c.fetchone()
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
240 else:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
241 fn = self._getblobfilename(blobid)
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
242 if os.path.isfile(fn):
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
243 with open(fn, "rb") as f:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
244 data = f.read()
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
245
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
246 if not data:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
247 raise RuntimeError("missing blob %s for fileid %s"
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
248 % (blobid, fileid))
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
249 if self.decompressor:
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
250 data = self.decompressor.decompress(data, 10**8)
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
251 return data
3b398a887b95 Use a sqlite3 database to store the blob data if available
Roy Marples <roy@marples.name>
parents: 93
diff changeset
252
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
253 def getmode(self, name, fileid):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
254 (commitid, blobid) = fileid
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
255 return self.filemodes[commitid][name]
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
256
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
257 def checkpoint_handler(self, cmd):
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
258 # This command means nothing to us
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
259 pass
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
260
57
e827bffa5fc5 Rename local variable committish to commitref.
Greg Ward <gward@intelerad.com>
parents: 55
diff changeset
261 def _getcommit(self, commitref):
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
262 """Given a mark reference or a branch name, return the
58
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
263 appropriate commit object. Return None if commitref is a tag
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
264 or a branch with no commits. Raises KeyError if anything else
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
265 is out of whack.
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
266 """
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
267 if commitref.startswith(b":"):
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
268 # KeyError here indicates the input stream is broken.
57
e827bffa5fc5 Rename local variable committish to commitref.
Greg Ward <gward@intelerad.com>
parents: 55
diff changeset
269 return self.commitmap[commitref]
58
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
270 elif commitref.startswith(self.tagprefix):
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
271 return None
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
272 else:
57
e827bffa5fc5 Rename local variable committish to commitref.
Greg Ward <gward@intelerad.com>
parents: 55
diff changeset
273 branch = self._getbranch(commitref)
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
274 if branch is None:
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
275 raise ValueError(b"invalid commit ref: %s" % commitref)
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
276
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
277 heads = self.branchmap.get(branch)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
278 if heads is None:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
279 return None
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
280 else:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
281 # KeyError here indicates bad commit id in self.branchmap.
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
282 return self.commitmap[heads[-1]]
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
283
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
284 def _getbranch(self, ref):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
285 """Translate a Git head ref to corresponding Mercurial branch
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
286 name. E.g. \"refs/heads/foo\" is translated to \"foo\".
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
287 Special case: \"refs/heads/master\" becomes \"default\". If
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
288 'ref' is not a head ref, return None.
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
289 """
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
290 prefix = b"refs/heads/"
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
291 if ref.startswith(prefix):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
292 branch = ref[len(prefix):]
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
293 if branch == b"master":
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
294 return b"default"
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
295 else:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
296 return branch
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
297 else:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
298 return None
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
299
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
300 def commit_handler(self, cmd):
39
177a133519bc Handle fixup branches for tag/branch creation better.
Greg Ward <greg-hg@gerg.ca>
parents: 37
diff changeset
301 # XXX this assumes the fixup branch name used by cvs2git. In
177a133519bc Handle fixup branches for tag/branch creation better.
Greg Ward <greg-hg@gerg.ca>
parents: 37
diff changeset
302 # contrast, git-fast-import(1) recommends "TAG_FIXUP" (not under
177a133519bc Handle fixup branches for tag/branch creation better.
Greg Ward <greg-hg@gerg.ca>
parents: 37
diff changeset
303 # refs/heads), and implies that it can be called whatever the
177a133519bc Handle fixup branches for tag/branch creation better.
Greg Ward <greg-hg@gerg.ca>
parents: 37
diff changeset
304 # creator of the fastimport dump wants to call it. So the name
177a133519bc Handle fixup branches for tag/branch creation better.
Greg Ward <greg-hg@gerg.ca>
parents: 37
diff changeset
305 # of the fixup branch should be configurable!
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
306 fixup = (cmd.ref == b"refs/heads/TAG.FIXUP")
39
177a133519bc Handle fixup branches for tag/branch creation better.
Greg Ward <greg-hg@gerg.ca>
parents: 37
diff changeset
307
58
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
308 if cmd.ref.startswith(self.tagprefix) and cmd.mark:
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
309 tag = cmd.ref[len(self.tagprefix):]
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
310 self.tags.append((tag, b':' + cmd.mark))
58
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
311
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
312 if cmd.from_:
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
313 first_parent = cmd.from_
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
314 else:
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
315 first_parent = self._getcommit(cmd.ref) # commit object
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
316 if first_parent is not None:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
317 first_parent = first_parent.rev # commit id
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
318
34
08e2157aaa9a Remove local fork of bzr-fastimport; use my fastimport library instead.
Greg Ward <greg-hg@gerg.ca>
parents: 33
diff changeset
319 if cmd.merges:
08e2157aaa9a Remove local fork of bzr-fastimport; use my fastimport library instead.
Greg Ward <greg-hg@gerg.ca>
parents: 33
diff changeset
320 if len(cmd.merges) > 1:
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
321 raise NotImplementedError("Can't handle more than two parents")
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
322 second_parent = cmd.merges[0]
44
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
323 else:
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
324 second_parent = None
44
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
325
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
326 if first_parent is None and second_parent is not None:
44
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
327 # First commit on a new branch that has 'merge' but no 'from':
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
328 # special case meaning branch starts with no files; the contents of
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
329 # the first commit (this one) determine the list of files at branch
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
330 # time.
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
331 first_parent = second_parent
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
332 second_parent = None
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
333 no_files = True # XXX this is ignored...
44
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
334
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
335 bfirst_parent = first_parent or b''
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
336 bsecond_parent = second_parent or b''
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
337 self.ui.debug(b"commit %s: first_parent = %s, second_parent = %s\n"
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
338 % (cmd, bfirst_parent, bsecond_parent))
44
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
339 assert ((first_parent != second_parent) or
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
340 (first_parent is second_parent is None)), \
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
341 (b"commit %s: first_parent == second parent = %s"
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
342 % (cmd, bfirst_parent))
44
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
343
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
344 # Figure out the Mercurial branch name.
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
345 if fixup and first_parent is not None:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
346 # If this is a fixup commit, pretend it happened on the same
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
347 # branch as its first parent. (We don't want a Mercurial
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
348 # named branch called "TAG.FIXUP" in the output repository.)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
349 branch = self.commitmap[first_parent].branch
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
350 else:
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
351 branch = self._getbranch(cmd.ref)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
352
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
353 commit_handler = HgImportCommitHandler(
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
354 self, cmd, self.ui)
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
355 commit_handler.process()
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
356 self.modified[cmd.id] = commit_handler.modified
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
357 self.filemodes[cmd.id] = commit_handler.mode
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
358 self.copies[cmd.id] = commit_handler.copies
16
234128693c29 Format the commit user as Full Name <user@example.com>
Paul Aurich <paul@darkrain42.org>
parents: 15
diff changeset
359
234128693c29 Format the commit user as Full Name <user@example.com>
Paul Aurich <paul@darkrain42.org>
parents: 15
diff changeset
360 # in case we are converting from git or bzr, prefer author but
234128693c29 Format the commit user as Full Name <user@example.com>
Paul Aurich <paul@darkrain42.org>
parents: 15
diff changeset
361 # fallback to committer (committer is required, author is
234128693c29 Format the commit user as Full Name <user@example.com>
Paul Aurich <paul@darkrain42.org>
parents: 15
diff changeset
362 # optional)
234128693c29 Format the commit user as Full Name <user@example.com>
Paul Aurich <paul@darkrain42.org>
parents: 15
diff changeset
363 userinfo = cmd.author or cmd.committer
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
364 if userinfo[0] == userinfo[1]:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
365 # In order to conform to fastimport syntax, cvs2git with no
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
366 # authormap produces author names like "jsmith <jsmith>"; if
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
367 # we see that, revert to plain old "jsmith".
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
368 user = userinfo[0]
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
369 else:
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
370 user = b"%s <%s>" % (userinfo[0], userinfo[1])
34
08e2157aaa9a Remove local fork of bzr-fastimport; use my fastimport library instead.
Greg Ward <greg-hg@gerg.ca>
parents: 33
diff changeset
371
62
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
372 text = cmd.message
34
08e2157aaa9a Remove local fork of bzr-fastimport; use my fastimport library instead.
Greg Ward <greg-hg@gerg.ca>
parents: 33
diff changeset
373 date = self.convert_date(userinfo)
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
374 parents = []
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
375 if first_parent:
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
376 parents.append(first_parent)
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
377 if second_parent:
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
378 parents.append(second_parent)
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
379
73
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
380 commit = common.commit(user, date, text, parents, branch,
a99e5c6c8e1c Fix compatibility with 4.6+
Daniel Johnson <daniel@daniel-johnson.org>
parents: 71
diff changeset
381 rev=cmd.id, sortkey=int(cmd.id[1:]))
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
382
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
383 self.commitmap[cmd.id] = commit
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
384 heads = self.branchmap.get(branch)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
385 if heads is None:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
386 heads = [cmd.id]
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
387 else:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
388 # adding to an existing branch: replace the previous head
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
389 try:
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
390 heads.remove(first_parent)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
391 except ValueError: # first parent not a head: no problem
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
392 pass
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
393 heads.append(cmd.id) # at end means this is tipmost
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
394 self.branchmap[branch] = heads
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
395 self.ui.debug(b"processed commit %s\n" % cmd)
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
396
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
397 def convert_date(self, c):
59
c82b6a84884f Convert timezones correctly.
Greg Ward <gward@intelerad.com>
parents: 58
diff changeset
398 res = (int(c[2]), -int(c[3]))
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
399 #print c, res
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
400 #print type((0, 0)), type(res), len(res), type(res) is type((0, 0))
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
401 #if type(res) is type((0, 0)) and len(res) == 2:
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
402 # print "go for it"
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
403 #return res
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
404 return b"%d %d" % res
92
efe678b023d3 Whitespace police
Roy Marples <roy@marples.name>
parents: 89
diff changeset
405
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
406 def reset_handler(self, cmd):
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
407 branch = self._getbranch(cmd.ref)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
408 if branch:
42
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
409 # The usual case for 'reset': (re)create the named branch.
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
410 # XXX what should we do if cmd.from_ is None?
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
411 if cmd.from_ is not None:
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
412 self.branchmap[branch] = [cmd.from_]
44
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
413 else:
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
414 # pretend the branch never existed... is this right?!?
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
415 try:
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
416 del self.branchmap[branch]
44
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
417 except KeyError:
61ff7b929cea Handle fixup commits and branch creation as produced by cvs2git.
Greg Ward <greg-hg@gerg.ca>
parents: 43
diff changeset
418 pass
42
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
419 #else:
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
420 # # XXX filename? line number?
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
421 # self.ui.warn("ignoring branch reset with no 'from'\n")
58
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
422 elif cmd.ref.startswith(self.tagprefix):
42
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
423 # Create a "lightweight tag" in Git terms. As I understand
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
424 # it, that's a tag with no description and no history --
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
425 # rather like CVS tags. cvs2git turns CVS tags into Git
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
426 # lightweight tags, so we should make sure they become
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
427 # Mercurial tags. But we don't have to fake a history for
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
428 # them; save them up for the end.
58
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
429 if cmd.from_ is not None:
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
430 tag = cmd.ref[len(self.tagprefix):]
c370e587f483 Handle commits commands that reference a tag better.
Greg Ward <gward@intelerad.com>
parents: 57
diff changeset
431 self.tags.append((tag, cmd.from_))
15
e19d2ce18eeb Fix reset_handler
Paul Aurich <paul@darkrain42.org>
parents: 11
diff changeset
432
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
433 def tag_handler(self, cmd):
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
434 pass
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
435
42
71f1e5ed6213 Convert "lightweight" tags, since that's how cvs2git handle CVS tags.
Greg Ward <greg-hg@gerg.ca>
parents: 41
diff changeset
436
1
9461f5c3a67c Actually imports something - incredible!
Paul Crowley <paul@lshift.net>
parents: 0
diff changeset
437 class HgImportCommitHandler(processor.CommitHandler):
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
438
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
439 def __init__(self, parent, command, ui):
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
440 self.parent = parent # HgImportProcessor running the show
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
441 self.command = command # CommitCommand that we're processing
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
442 self.ui = ui
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
443
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
444 # Files changes by this commit as a list of (filename, id)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
445 # tuples where id is (commitid, blobid). The blobid is
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
446 # needed to fetch the file's contents later, and the commitid
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
447 # is needed to fetch the mode.
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
448 # (XXX what about inline file contents?)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
449 # (XXX how to describe deleted files?)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
450 self.modified = []
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
451
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
452 # mode of files listed in self.modified: '', 'x', or 'l'
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
453 self.mode = {}
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
454
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
455 # dictionary of src: dest (renamed files are in here and self.modified)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
456 self.copies = {}
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
457
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
458 # number of inline files seen in this commit
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
459 self.inlinecount = 0
92
efe678b023d3 Whitespace police
Roy Marples <roy@marples.name>
parents: 89
diff changeset
460
93
2ce33511de87 hgimport: Pass on deleteall_handler
Roy Marples <roy@marples.name>
parents: 92
diff changeset
461 def copy_handler(self, filecmd):
2ce33511de87 hgimport: Pass on deleteall_handler
Roy Marples <roy@marples.name>
parents: 92
diff changeset
462 self.copies[filecmd.src_path] = filecmd.dest_path
2ce33511de87 hgimport: Pass on deleteall_handler
Roy Marples <roy@marples.name>
parents: 92
diff changeset
463
2ce33511de87 hgimport: Pass on deleteall_handler
Roy Marples <roy@marples.name>
parents: 92
diff changeset
464 def delete_handler(self, filecmd):
2ce33511de87 hgimport: Pass on deleteall_handler
Roy Marples <roy@marples.name>
parents: 92
diff changeset
465 self.modified.append((filecmd.path, None))
2ce33511de87 hgimport: Pass on deleteall_handler
Roy Marples <roy@marples.name>
parents: 92
diff changeset
466
2ce33511de87 hgimport: Pass on deleteall_handler
Roy Marples <roy@marples.name>
parents: 92
diff changeset
467 def deleteall_handler(self, filecmd):
2ce33511de87 hgimport: Pass on deleteall_handler
Roy Marples <roy@marples.name>
parents: 92
diff changeset
468 pass # Fossil fastexport will hit this with no parents
2ce33511de87 hgimport: Pass on deleteall_handler
Roy Marples <roy@marples.name>
parents: 92
diff changeset
469
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
470 def modify_handler(self, filecmd):
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
471 if filecmd.dataref:
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
472 blobid = filecmd.dataref # blobid is the mark of the blob
11
9e9c215fcbd8 Handle blobs in the fast-import stream.
Greg Ward <greg-hg@gerg.ca>
parents: 10
diff changeset
473 else:
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
474 blobid = b"%s-inline:%d" % (self.command.id, self.inlinecount)
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
475 assert filecmd.data is not None
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
476 self.parent.writeblob(blobid, filecmd.data)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
477 self.inlinecount += 1
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
478
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
479 fileid = (self.command.id, blobid)
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
480
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
481 self.modified.append((filecmd.path, fileid))
62
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
482 if stat.S_ISLNK(filecmd.mode): # link
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
483 mode = b'l'
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
484 elif filecmd.mode & 0o111: # executable
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
485 mode = b'x'
62
76bd0ea8add3 Update to cope with API differences.
Jelmer Vernooij <jelmer@samba.org>
parents: 55
diff changeset
486 elif stat.S_ISREG(filecmd.mode): # regular file
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
487 mode = b''
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
488 else:
74
a4f13dc5e3f7 Support Mercurial 5.6 and Python-3.6
Roy Marples <roy@marples.name>
parents: 73
diff changeset
489 raise RuntimeError(b"mode %s unsupported" % filecmd.mode)
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
490
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
491 self.mode[filecmd.path] = mode
0
d107c6d36780 Add the start of the hg fastimport command
Paul Crowley <paul@lshift.net>
parents:
diff changeset
492
47
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
493 def rename_handler(self, filecmd):
7ff36dc9f0b1 Massive rework to use infrastructure provided by convert extension.
Greg Ward <greg-hg@gerg.ca>
parents: 46
diff changeset
494 # copy oldname to newname and delete oldname
71
5dc3ab8142d5 Add patch to fix rename handling by correcting the used attribute names.
Ralph Meijer <ralphm@ik.nu>
parents: 68
diff changeset
495 self.copies[filecmd.new_path] = filecmd.old_path
5dc3ab8142d5 Add patch to fix rename handling by correcting the used attribute names.
Ralph Meijer <ralphm@ik.nu>
parents: 68
diff changeset
496 self.modified.append((filecmd.old_path, None))